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ABSTRACT 


This  Satisfy  investigated  the  preference  of  two  types 
of  computer  animation  graphics  displays  --  two  and  three 
dimensional.  A  preference  was  determined  based  on  how  well 
they  supported  the  decision  making  process  within  the  Air 
Base  Operability  orgarization  (ABO).  The  procedure  used 
was  the  true  experiment  method  to  compare  the  two  different 
display  types  and  the  standard  Air  Base  Operability  display 
to  see  if  there  was  a  significant  difference.  The 
experiment  was  conducted  on  a  personal  home  computer  with  a 
color  graphics  capability.  Subjects  were  randomly  assigned 
to  one  of  the  three  groups  using  a  particular  display. 

The  basic  objective  of  the  experiment  scenario  was  to  find 
damaged  areas  inflicted  on  a  simulated  Air  Force  base.  The 
results  from  the  experiment  indicated  no  significant 
difference  between  the  two  dimensional  and  three 
dimensional  graphics  displays.  However,  there  was  a 
significant  difference  between  the  three  dimensional 
display  and  the  standard  ABO  display.  The  conclusions 
drawn  from  this  experiment  shows  that  not  enough  data  was 
collected  from  the  experiment  to  prove  a  significant 
difference  between  the  displays.  The  primary 
recommendation  made  by  this  researcher  is  to  reconduct  the 
experiment  with  a  more  substantial  sample  population  from 


each  group. 
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ASSESSING  THE  PREFERENCE  OF  TWO  OR  THREE  DIMENSIONAL 
COMPUTER  GRAPHICS  ANIMATION  DISPLAYS  ON  THE 
AIR  BASE  OPERABILITY  DECISION  MAKING  PROCESS 


I.  Introduction 


Overview 

Chapter  one  provides  background  information  on  the 
general  issue  of  determining  the  preference  between  two 
types  of  computer  graphics  animations.  This  chapter  will 
also  state  and  explain  the  study’s  specific  problem 
statement,  its  purpose  and  justification,  its  specific 
research  objectives  and  the  scope  of  the  application  using 
computer  graphics  animation. 

Background 

Over  the  last  two  decades,  computer  graphics  has 

improved  its  development  from  simple  displays  of 

monochromatic  mathematical  wire  diagrams  to  a  very 

sophisticated  system  of  multiple  color  processed  images 

with  filled  in  faces.  In  these  newer  systems,  the  operator 

can  now  interact  with  these  display  devices  to  better 

assist  and  improve  programming  and  decision  making.  As  one 

researcher,  Brian  Berry  states: 

The  1960s  have  been  characterized  as  the  era  of 
computer  numbers,  the  70s  as  the  era  of  computer 
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words,  and  the  80s  as  the  era  of  computer  graphics. 
Computer  graphics  has  extended  traditional  data 
processing  methods  by  providing  ways  of  communicating 
information  pictorially.  (3:xv) 

Interactive  computer  graphics  is  the  first  step  toward 
a  more  common  man-machine  interaction.  Computer 
communication  has  developed  into  a  technology  for 
converting  virtually  unlimited  information  into  visual 
images.  One  researcher  defines  computer  graphics  as  ‘a 
consciously  managed  and  documented  technology  directed 
toward  communicating  information  accurately  and 
descriptively"  (10:398). 

When  the  first  personal  computers  were  developed  there 
arose  great  expectations  as  to  how  the  computer  could 
improve  the  display  of  information.  In  earlier  years, 
computer  graphics  were  expensive  due  to  labor  costs  for 
data  input,  the  time  consumed  with  generating  the  graphics, 
the  rate  of  calculating  and  manipulating  the  data,  and  the 
large  memory  capacity  and  disk  space  required 
(22:21-24).  Now,  inexpensive  personal  computers  can  begin 
to  fulfill  those  expectations.  Low-cost  microcomputers, 
such  as  one  owned  by  this  researcher,  allow  graphics  to  be 
a  generally  available  resource.  Faster  microprocessors 
have  been  developed  which  generate  a  computer  graphic 
animation  in  1/lOth  of  the  time  formerly  required. 
Additionally,  file  compression  methods  have  made  animations 
easier  to  store  on  disk.  Angell  states: 
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Until  recently,  all  but  the  most  trivial  computer 
graphics  was  the  province  of  specialized  research 
groups.  Now  with  the  introduction  of  reasonably 
priced  graphics  devices,  the  subject  will  reach  many 
more  users  and  its  full  potential  can  realized. 
Computer-produced  pictures  always  impress  the  layman, 
and  the  'mysterious'  techniques  used  for  drawing  them 
have  gained  a  [false]  reputation  for  complexity. 

( 1  :  V  i  i  ) 

In  part,  computer  graphics  has  entered  almost  every 
field  of  endeavor  including  business  and  industry. 

Research  must  continue  to  pursue  how  much  computer  graphics 
can  enhance  organizations.  Therefore,  it  is  necessary 
first  explain  why  computer  graphics  has  entered  into  the 
management  field. 


Computer  Graphics  In  Management 

The  prominent  reason  for  this  increased  usage  of 

computer  graphics  in  management  is  due  to  the  increased 

utilization  within  the  decision-making  area  (3:xvi, 

7:463,  8:40,  16:764,  20:757).  Computer  graphics  has  been 

found  by  decision  makers  as  an  available  and  powerful 

device  for  passing  information  to  people  in  any  kind  of 

circumstance.  Beyond  the  simple  display  of  the  graphic 

image  itself,  these  pictures  can  be  used  in  the  context  of 

an  elaborate  information  system  to  bring  about  augmented 

analytic  and  descriptional  capacities  for  users.  One 

example  is  illustrated  by  Berry: 

A  geographic  information  system  (GIS)  allows  a  user 
not  only  to  display  results  pictorially,  but  also  to 
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perform  a  number  of  analytic  operations  on  the  data, 
interface  other  existent  data  bases,  and  enjoy  a 
general-purpose  data-creat ion  capability.  (3:xvi) 

One  researcher’s  (18:57-67)  investigation  concerning 

graphics  usage  noticed  a  significant  trend  in  the  use  of 

computer  graphics  by  users  in  organizations.  Other 

surveys,  performed  by  Lehman  in  1984  and  1986,  determined 

the  impact  of  computer  graphics  on  the  MIS.  From  his  1986 

research  he  concludes: 


At  this  time,  microcomputer-based  graphics  and 
mainframe-based  graphics  are  not  just  technological 
alternatives.  They  are  not  integrated  into  existing 
information  system  to  the  same  degree  and  their  users 
report  different  advantages  and  disadvantages  which 
mirror  the  distinctions  between  MIS-oriented  and 
user-oriented  systems  or  between  organizational  and 
personal  computing.  (17:139) 

Therefore,  prior  to  1986,  graphics  was  not  a  viable 
decision  support  system  alternative.  Since  1984,  however, 
there  has  been  a  rapid  growth  of  computer  graphics  in  areas 
such  as  the  decision-making  process.  One  reason  for  this 
rapid  growth  has  been  the  availability  of  environmental 
data  for  regional  planning  and  civil  engineering 
(3:66).  This  is  just  one  example  and  other  examples  will 
be  illustrated  in  the  literature  review  chapter.  One 
particularly  important  reason  for  using  computer  graphics 
in  the  decision-making  arena  is  that  it  saves  time;  a  most 
valuable  commodity.  It  is  extremely  easy  for  computer 
graphic  systems  to  manipulate  various  information  into 
charts,  maps,  and  graphs.  The  building  of  various  charts, 
maps,  and  graphics  assist  in  the  explanation  and 
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dissemination  of  complex  sets  or  groups  of  data.  Complex 
sets  of  information  can  be  comprehended  more  easily  when 
visually  represented.  Additionally,  tendencies  and 
peculiarities  in  the  information  data  are  often  perceived 
more  easily  when  it  is  generated  in  a  graphical  image  as 
opposed  to  displaying  the  Information  through  tables  and 
charts.  This  ability  to  spot  alterations  in  the  data  is 
needed  by  decision  makers  to  better  describe,  sort  out, 
validate  and  exclude  any  wrong  information.  Through 
computer  graphics,  decision  makers  are  also  provided  with 
the  ability  to  ask  questions  to  test  alternative  scenarios 
and  then  depict  the  results  graphically,  in  a  quick  and 
efficient  manner.  Decision  makers  will  turn  toward  such 
systems  for  assistance  as  these  information  systems  become 
more  generic  and  the  man/machine  interface  becomes  simpler. 

Currently,  there  two  different  means  of  displaying 
graphics.  These  two  means  are  two  dimensional  and  three 
dimensional  computer  graphics,  and  both  will  be  thoroughly 
explained  in  the  literature  review  chapter. 


Specific  Problem  Statement 

Organizations,  when  in  need  of  improvement,  can  turn 
to  computers  for  assistance  in  their  decision-making 
process,  and  a  portion  of  that  assistance  could  be  derived 
from  computers  graphics.  The  addition  of  computer  graphics 
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technology  can  increase  the  decision-making  abilities  of 

the  managers.  Now  that  an  organization  has  the  means,  what 

choices  does  it  have?  What  type  of  graphic  display  will 

best  assist  an  organization  with  its  decision-making 

process?  While  there  are  many  solutions  available,  the 

wrong  choice  is  assured  to  be  costly. 

In  order  to  determine  what  graphic  display  method 

could  improve  the  decision-making  process  for  a  particular 

organization,  it  is  necessary  to  conduct  research  using  a 

specific  organization's  decision-making  environment. 

The  complex  battlefield  environment,  with  rapid 

tactical  decisions  made  by  military  commanders  on  the  basis 

of  multiple  sets  of  rapidly  changing  data,  is  one  such 

environment.  This  environment  causes  extreme  difficulties, 

since  decisions  are  based  on  data  presented  and  absorbed 

through  using  conventional  maps,  charts,  photographs,  and 

other  graphical  aids  such  as  plotting  boards.  Such  media 

are  inflexible,  difficult  to  update,  and  hard  to  Interpret 

if  cluttered  with  data  not  relevant  to  the  decision-making 

process.  Using  either  two  or  three  dimensional  computer 

animation  graphics  might  prove  beneficial.  The  need  for 

new  techniques  for  which  the  application  of  computer 

graphics  might  be  worthwhile,  is  suggested  by  Schachter: 

A  hostile  environment  may  include  as  many  as  30,000 
potential  targets  Including  such  major  items  as  tanks, 
armored  personnel  carriers,  artillery  tubes,  air 
defense  systems,  command  posts,  and  logistics  centers. 
Ox  the  total ,  about  1 ,000  of  these  may  be  high 
priority  targets.  It  is  immediately  apparent  that 
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there  is  a  significant  disparity  between  the  large 
number  of  tactical  elements  present  and  the  major 
items  of  concern  to  the  commander.  Traditional 
methods  involving  the  use  of  hand-delivered  paper 
reports  and  paper  maps  with  Plexiglas  overlays  for 
grease  pencil  annotation  are  not  well-suited  for  a 
rapid  flow  and  culling  of  information.  New  techniques 
are  needed  to  enable  commanders  to  absorb  and  analyze 
the  mass  of  information  they  receive  so  that  they  can 
make  decisions  which  are  both  timely  and  correct. 
{27;xiii) 

Therefore,  one  possible  decision-making  arena  in  the 
Air  Force  which  could  benefit  from  computer  graphics  is  the 
Air  Base  Operability  (ABO)  damage  assessment  organization. 
The  ABO  function  is  to  accurately  record  and  report  all 
damage  inflicted  from  enemy  attack  on  an  airfield  and 
surrounding  area.  The  ABO  decision  maker  then  decides 
whether  those  locations  assessed  are  worthy  of  continued 
use  or  requires  repair.  This  researcher  proposes 
investigating  the  decision  making  process  utilizing  either 
two  or  three  dimensional  computer  animation  graphics  within 
the  Air  Base  Operability  (ABO)  decision-making  environment. 
The  ABO  environment  poses  as  an  excellent  environment  in 
which  to  conduct  the  proposed  research  and  compare 
different  computer  graphics  techniques.  First,  because  the 
ABO  environment  currently  utilizes  graphics  data  as  input 
based  on  damaged  areas  to  buildings,  runways,  etc.,  to  make 
important  decisions.  Secondly,  the  Air  Force  is  currently 
developing  a  Future  Airfield  Damage  Assessment  System 
(ADAS),  as  explained  in  the  literature  review  chapter,  to 
assist  in  processing  airfield  damage  data.  Thus  the 
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prospect  of  conducting  research  to  determine  which  type  of 
graphics  display  for  the  ABO  decision  maker  is  important. 
The  correct  graphics  display  would  better  assist  those  who 
assess  the  damage  and  conduct  briefings  to  keep  commanders 
informed  on  he  situation  to  make  important  decisions.  An 
in  depth  explanation  of  how  the  ABO  operation  is  conducted 
and  what  its  mission  consists  of  is  explained  in  chapter 
II.  Therefore,  the  ABO  operation  makes  an  ideal  case  to 
use  for  research  to  determine  which  computer  graphic 
animation  would  most  benefit  the  ABO  decision-making 
process.  An  understanding  of  how  a  specific  display 
impacts  the  ABO  decision-making  process  would  further 
assist  research  on  computer  displays  for  additional 
decision  making  arenas.  The  problem  statement  for  this 
study  is: 

Which  type  of  computer  animation  graphic  display, 
either  two  or  three  dimensional,  would  be  most  beneficial 
for  any  decision  making  process? 

Purpose 

The  purpose  of  this  study  is  to  investigate  the  impact 
of  two  and  three  dimensional  computer  graphics  displays  on 
the  ABO  decision-making  process.  This  investigation  will 
attempt  to  determine  which  graphic  display  best  improves 
the  ABO  decision  making  process  and  will  examine  the 
differences  between  the  two  types  of  graphics  with  respect 
to  the  existing  method  used. 
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Specific  Objective 


The  specific  objective  of  this  research  is  to  develop 
and  test  two  different  types  of  computer  animation  graphic 
displays.  The  first  consists  of  a  two  dimensional  graphic 
animation  and  the  second  consists  of  a  three  dimensional 
graphic  animation  display.  These  two  displays  will  then  be 
tested  in  an  experiment  given  identical  criteria,  and  their 
results  will  be  compared  by  analysis  of  variance  to 
determine  if  there  are  any  differences  from  the  current 
display  method. 

Hypothesis  and  Investigative  Questions 

The  hypothesis  that  this  research  hopes  to  prove  is 
that  there  is  a  definable  difference  between  the  two 
computer  graphics  displays  and  that  three  dimensional 
computer  animation  graphics  will  best  improve  any 
decision-making  process.  The  above  specific  objective  will 
be  met  by  answering  the  following  investigative  questions; 

1.  Is  there  a  significant  difference  with  using  two 
dimensional  or  three  dimensional  computer  animation 
graphics  in  the  ABO  decision-making  process? 

2.  What  ABO  information  can  or  cannot  be  presented 
with  each  type  of  animation  graphic  display? 

3.  What  time  constraints  are  applicable  with  the 
production  two  dimensional  or  three  dimensional  animation 
graphics  for  the  decision  maker? 
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4.  What  finances  are  associated  with  the  acquisition 

of  a  computer  graphics  display  system  and  will  it  be  cost 

ef  f ective? 

Investigative  questions  1  and  2  will  be  answered, 
through  the  methodology  and  analysis  of  the  data  discussed 
in  Chapters  III  and  IV.  Investigative  questions  3  and  4 
will  be  answered  in  the  literature  review  in  Chapter  II. 

Scope 

The  scope  of  this  research  will  cover  the  best  set  of 

criteria  necessary  for  the  decision-making  process.  In 

order  to  determine  the  effectiveness  of  the  computer 
animation  graphic  display,  one  specific  functional  area, 
such  as  Air  Base  Operability,  was  chosen.  The  decision 
support  area  will  cover  the  damage  assessment  phase  during 
the  post-attack  period  on  an  air  force  base.  Further 
research  into  other  areas  should  be  accomplished  to 
determine  if  they,  too,  may  also  be  benefited  by  the 
addition  of  computer  graphics.  However,  for  purposes  of 
this  research,  only  the  damage  assessment  phase  will  be 
used . 
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Literature  Review 


II  . 


Overview 

There  are  two  different  types  of  computer  graphics 
available  for  managerial  functions  and  decision  support 
systems  -  two  dimensional  or  three  dimensional  computer 
generated  images  or  animations.  The  purpose  of  this 
literature  review  is  to  discuss  the  capabilities  of  two 
dimensional  and  three  dimensional  computer  graphics  in 
management  and  prior  research  which  has  been  conducted. 
First,  an  overview  of  some  of  the  trends  in  graphics 
definition  of  two  dimensional  and  three  dimensional 
graphics  as  defined  by  other  researchers  will  be  discussed 
Following  this,  examples  of  how  computer  graphics  have 
enhanced  certain  applications  in  the  decision  support  area 
will  be  presented.  And  finally,  this  review  will  examine 
how  the  Air  Base  Operability  decision-making  process 
functions  during  its  damage  assessment  phase  during  the 
post-attack  period. 

Computer  Graphics  Trends 

Numerous  researchers  have  noted  a  tremendous  growth 
with  computer  graphics.  As  mentioned  before,  this  growth 
seems  to  exist  because  of  the  decreasing  costs  for  compute 
systems.  This  is  verified  by  DeSanctls  when  he  stated: 


The  computer  graphics  industry  is  estimated  to  be 
growing  at  an  annual  rate  of  60  to  70  percent. 

Rapidly  declining  hardware  costs,  flexible  easy-to-use 
software,  and  increased  availability  of  graphics 
technology  for  microcomputers  and  office  workstations 
account  for  this  trend.  (7:463) 

In  1989,  Lehman  mentioned  in  the  previous  chapter  on 
graphics  trends,  conducted  an  analysis  of  the  two  surveys 
he  did  earlier  and  declared  some  basic  findings  for 
computer  graphics: 

Considerable  penetration  has  occurred  since  1984, 

Graphics  have  become  better  integrated  into  the 
overall  organizational  information  system. 

Factors  in  the  success  of  computer  graphics  remain 
the  same:  ease  of  use  and  data  availability,  and 

Applications  have  shifted  from  support  to  decision 
making  to  support  of  communications.  (18:67) 

DeSanctis  and  others  suggested,  however,  that  ‘little 

research  is  available  on  the  impact  of  graphics  on  decision 

making ...‘  (7 : 463)  Therefore,  investigation  of  the  research 

on  computer  graphics  is  in  order,  since  decision  support 

organizations  are  starting  to  realize  the  importance  of 

computer  graphics-based  systems.  Then,  it  will  be  necessary 

to  review  important  research  findings  of  how  computer 

graphics  have  benefited  decision  making. 


Computer  Graphics  Research 

Research  developments  have  concluded  that  certain 
computer  graphics  outputs  are  helpful  in  the  decision 
making  process  (7:482,  8:44,  16:772).  One  example  is  an 


12 


experiment  conducted  by  Izak  Benbasat  pertaining  to  color 
graphics  (2:1104).  Benbasat  concluded  that  color  graphics, 
as  opposed  to  tabular  display  outputs,  played  an  important 
role  in  the  decision-making  process,  but  only  to  a  certain 
extent.  The  limitation  was  that  while  color  graphics  was 
excellent  in  making  short,  quick  judgements,  for  a  long 
term  scenario,  however,  tabular  outputs  were  better.  An 
important  finding  of  the  experiments  was  that  the  benefits 
of  color-coding  were  more  strongly  associated  with 
graphical  rather  than  tabular  display  formats.  The  most 
important  finding  was  the  influence  of  color  in  relation  to 
interaction  with  the  cognitive  style  of  the  decision  maker 
(2:1102).  Color  also  appeared  to  assist  the  decision  maker 
under  a  time  constraint  or  during  the  early  stages  of 
problem  solving.  Also  evident  was  that  graphical 
presentations  enhanced  decision-making  speed,  but  only  if 
the  report  was  designed  in  a  form  which  directly  assisted 
with  the  problem  solving  heuristic.  While  graphs  were 
found  most  helpful  in  calculating  and  comparing  values  of  a 
general  region,  they  were  not  helpful  for  exact  data  values 
required  for  optimization. 

Computer  Graphics  Usage 

Computer  graphics  have  come  a  long  way  from  their 
first  usage.  Computer  graphics  were  first  used  as  an  aid 
to  designing  objects.  Computer-aided-design  (CAD)  has 
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provided  a  means  for  creating  all  types  of  architectural 
plans  and  drawings.  For  example,  CAD  programs  can  describe 
any  machine  part  in  three  dimensions.  Another  example  is 
that  architects  are  using  building  layouts  produced  by  CAD 
methods  as  design  aids  for  housing  sales  (23.143-150).  Two 
dimensional  floor  plans  are  used  to  design  room 
arrangement,  door  and  window  placement,  or  the  location  of 
various  facilities.  Three  dimensional  displays  are  used  to 
view  the  appearance  of  a  single  building  design  or  to  test 
the  appearance  of  a  complex  arrangement  of  buildings  (a 
mall,  university  campus,  or  an  air  force  base). 

Computer  graphics  techniques  are  also  used  by  artists. 
Animated  cartoons  and  movies  are  produced  with  the  aid  of 
graphics  displays.  One  example  is  'TRON',  one  of  the  first 
motion  pictures  made  which  used  computer  generated  images 
for  most  of  its  backgrounds  and  special  effects 
(27:vi).  These  effects  appeared  almost  life-like,  giving 
the  audience  a  better  understanding  of  what  it  could  be 
like  inside  a  computer.  Workers  in  many  areas  are  relying 
on  computer  graphics  as  an  important  Integrated  tool  for 
studying  the  characteristics  of  systems. 

In  addition  to  models,  computer  generated  graphs  and 
charts  are  used  to  interpret  mathematical  relationships  or 
to  study  trends  in  the  behavior  of  systems.  These  graphs 
produced  from  the  graphics  data  inputted  into  computers  can 
take  a  variety  of  forms.  Two  dimensional  weather  maps,  for 
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example,  can  be  constructed  by  graphics  systems  from  data 
supplied  by  satellite  observation  stations.  In  another 
example  (3:xix),  pressure  or  temperature  variations  over 
geographical  areas  was  compared  in  three  dimensional  plots. 

Similar  techniques  are  used  in  many  business  and 
government  applications.  These  applications  represent  one 
of  the  largest  groups  of  computer  graphics  users.  Various 
types  of  line  graphs  and  bar  charts  are  used  for 
summarizing  financial  and  statistical  data.  While  three 
dimensional  pictures  and  graphs  are  VAS»d  to  show  multi 
dimensional  information,  these  pictures  and  graphs  are 
often  generated  for  managerial  reports,  for  consumer 
information  bulletins,  or  for  visual  aids  to  be  used  during 
presentations.  Besides  the  basic  uses  of  line,  bar,  and 
charts  used  to  display  information,  there  are  other  uses  in 
computer  graphics  which  fully  utilize  its  potential 
app 1 i cations . 

In  the  1970s,  computer  graphics  were  just  an 
alternative  means  to  huge  amounts  of  hole  punch  cards  and 
paper  output  (14:146).  Even  as  late  as  1982,  there  were 
only  three  types  of  monitor  display  technologies  available 
for  graphics  (22:6),  which  were  RGB,  composite  and 
separated.  Only  eight  colors  were  available  for  graphics 
pictures  and  the  memory  to  display  graphic  animations  was 
exhaustive  and  not  enough  for  most  computers  consisting  of 
only  512  kilobytes  of  Random  Access  Memory  (RAM).  Today, 
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graphics  can  consist  of  4096  or  more  different  colors 
displayed  all  at  once.  Additionally,  there  is  a  new 
technique  called  color  cycling,  which  imitates  animation  of 
motion  and  uses  small  amounts  of  memory.  Computer  graphics 
has  made  a  tremendous  leap  ahead  from  its  first  usage. 

Development  of  microcomputer  technology  has  led  to  the 
design  of  small-scale  computer  systems  with  a  broad  range 
of  graphics  capabilities.  Desk-top  computers  can  be 
configured  for  graphics  uses  in  the  office,  in  research  and 
design  and  in  schools  and  as  personal  computers.  Their  low 
cost,  compact  size  and  versatility  make  them  highly 
suitable  for  a  wide  variety  of  graphics  applications.  The 
research  conducted  and  the  graphics  displays  shown  for  this 
thesis  utilizes  a  personal  computer,  the  Amiga  2000.  But 
to  utilize  some  of  the  new  potentials  of  computer  graphics, 
an  understanding  is  necessary  of  what  different  types  of 
computer  graphics  there  are. 

Computer  Graphics  Systems 

The  basic  components  of  a  computer  graphics  system 
comprise  of  a  CPU  (Computer  Processing  Unit),  keyboard, 
video  monitor,  and  the  necessary  graphics  programs.  Most 
computers  additionally  allow  peripheral  equipment  such  as 
digitizers,  as  input  or  output  devices. 

Because  computer  graphics  are  mainly  involved  with  the 
video  display  monitor,  it  is  an  important  component  to 
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understand.  All  computer  monitors  are  some  type  of  Cathode 
Ray  Tube  (CRT) .  These  CRTs  can  be  designed  to  function  as 
random-scan  or  raster-scan  devices.  The  type  of  function 
used  makes  a  difference  of  how  clearly  a  picture  is  shown. 
Each  has  its  own  advantages  and  disadvantages  making  the 
picture  look  better  or  worse  depending  on  the  cost.  A  home 
TV  is  an  example  of  a  raster-scan  CRT.  Techniques  other 
than  the  basic  CRT  design  have  been  used  for  constructing 
video  displays.  These  devices  include  plasma  panel 
displays  (26:3-29),  laser  scan  displays,  light-emitting 
diodes  (LEDs),  and  liquid-crystal  (LCD)  systems.  Each 
presents  a  different  clarity  in  the  graphic  picture  being 
displayed.  With  an  understanding  of  the  video  display 
monitor  in  hand,  let  us  turn  now  to  how  a  graphic  picture 
is  presented  on  the  monitor,  and  what  a  picture  is 
comprised  of . 

Display  Graphics  Concepts 

Graphics  systems  are  often  characterized  by  the  number 
of  available  colors  and  by  the  number  of  points  -  referred 
to  as  the  resolution  --  that  can  be  plotted  on  the  screen. 
Resolution  for  graphics  systems  is  given  in  terms  of  the 
number  of  horizontal  and  vertical  display  points  available. 
These  numbers  are  set  by  the  graphics  computer  system,  but 
are  also  dependent  on  the  type  of  video  display  device 
used.  Systems  using  a  standard  TV  monitor  have  an  upper 
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limit  of  about  512  points  horizontally  and  256  points 
vertically.  Special  graphics  terminals  and  monitors  are 
capable  of  a  2000  by  2000  pixel  resolution  today. 

Each  character  displayed  on  a  video  screen  takes  up  a 
small  rectangular  space.  This  space  is  divided  even 
further  into  smaller  rectangles.  These  smaller  rectangles 
are  called  pixels  (13:30).  A  simple  character  can  be 
several  pixels  in  size  depending  on  the  computer  system. 
When  a  computer  graphics  program  is  active,  it  accesses  the 
pixels  directly.  The  graphics  pixels  are  simply  "turned 
on"  when  needed  and  a  specific  color  is  chosen.  Pixel 
locations  are  referenced  by  a  coordinate  system  (X,Y).  The 
horizontal  distance  is  measured  by  X,  and  the  vertical 
distance  by  Y  and  start  from  the  top  left  of  the  display 
monitor  going  from  left  to  right,  top  to  bottom.  The 
number  of  pixels  plotted  per  centimeter  (cm)  is  referred  to 
as  resolution,  and  is  commonly  used  to  describe  the 
clearity  of  a  graphic  image.  Display  monitors  have  the 
ability  to  show  different  resolutions,  depending  on  the 
graphic  program,  but  how  large  a  display  monitor  one  might 
have  has  little  importance  on  resolution.  It  depends  only 
on  the  capability  of  the  monitor  to  change  resolutions. 

The  different  resolution  modes  are  described  in  three 
categories  -  Low  (displaying  fewer  points) ,  Medium  and  High 
resolution  (showing  the  most  pixels/cm) . 
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With  the  basic  components  of  a  computer  graphics 
system  explained,  it  is  now  essential  to  explore  the  two 
different  types  of  computer  graphics  generated  --  two  and 
three  dimensional . 

Two  Dimensional  Graphics 

Two  dimensional  graphics  are  the  viewing  of  a  computer 
generated  picture  with  only  2  degrees  of  freedom 
(15:42).  This  freedom  consists  of  its  axis  running 
horizontally  and  the  Y  axis  running  vertically  on  the 
screen.  To  create  a  two  dimensional  graphic  picture  is  to 
first  determine  which  parts  of  the  model  or  object  are  to 
be  displayed  and  whether  or  not  any  overall  adjustments  to 
the  picture  are  to  be  made  as  a  whole.  Eventually,  when 
creating  a  two  dimensional  graphic  picture,  the  developer 
must  decide  which  side  of  an  item  the  viewer  is  to  see  and 
which  other  items  lying  within  the  area  are  drawn. 

Examples  of  two  dimensional  pictures  would  be  top  views  or 
side  views,  such  as  maps  or  photographs.  Finally,  the  user 
must  decide  on  the  details  concerning  the  appearance  of  the 
object,  such  as  size,  color,  or  location  on  the  screen. 

When  drawing  the  picture,  several  effects  can  be  used 
to  create  realism.  A  two  dimensional  object  can  appear  to 
be  three  dimensional,  but  do  not  be  fooled.  Through  a 
technique  called  anti-aliasing,  a  graphic  image  can  be  made 
to  seemingly  stick  out  of  the  picture.  Anti-aliasing  is 
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the  process  of  adding  several  grey  scale  colors  to  the  side 
of  the  image  produced  starting  with  a  light  grey  to 
eventual  black,  creating  the  effect  that  the  object  appears 
to  stand  apart  from  its  background.  Anti-aliasing  can  also 
be  used  to  give  an  object  more  detail  and  not  seem 
cartoonish.  Using  darker  shades  of  the  same  color,  an 
object  can  have  Indentations  and  dimensions  within  itself 
again,  adding  more  realism.  Another  effect  which  is  very 
useful  in  two  dimensional  graphics  is  color  cycling.  Color 
cycling  involves  shifting  a  designated  color  over  one  pixel 
or  color  address  to  another  (28:215).  If  the  color  address 
range  is  not  too  extreme,  the  effect  gives  the  graphic 
picture  a  simulated  animation  effect.  An  example  would  be 
to  cycle  a  torch  flame.  The  color  range  would  be  from 
yellow  to  red  and  the  effect  would  give  the  torch  movement. 

There  are  several  benefits  to  using  two  dimensional 
graphics  as  a  type  of  decision  support  display.  One 
benefit  is  the  storage  size.  Two  dimensional  graphics  have 
relatively  small  file  sizes,  and  therefore  take  up  very 
little  disk  space.  If  a  user  is  required  to  prepare  a 
briefing  consisting  of  multiple  computer  graphics  pictures, 
two  dimensional  graphics  pictures  would  take  less  disk 
storage  room,  thus  more  pictures  could  be  presented. 

Another  benefit  is  the  ease  of  digitizing  an  image  from  a 
photograph  or  video  picture  to  a  two  dimensional  graphic. 
This  allows  for  any  user  to  become  an  artist  simply  by 
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pointing  a  video  camera  at  an  object,  hitting  a  key,  and 
through  a  computer  program  the  object  picture  is 
transferred  into  computer  memory  and  displayed  on  the 
monitor.  This  image  can  then  be  readily  changed  with 
respect  to  color,  size,  and  even  aspect  using 
anti-aliasing. 

There  are  some  disadvantages  to  two  dimensional 
graphics.  One  disadvantage  is  trying  to  get  more 
information  from  the  graphic  than  is  available.  If  a  user 
wanted  to  see  the  other  side  of  the  object,  it  would  be  an 
impossible  task,  unless  a  totally  separate  picture  had  been 
generated.  Another  disadvantage  is  that  two  dimensional 
graphic  pictures  tend  to  give  more  information  than  is 
believed  (28:217).  Two  dimensional  pictures  deceive  the 
user  to  interpret  more  information  about  the  picture. 
Additionally,  user  freedom  to  move  around  the  graphic 
picture  is  limited.  A  two  dimensional  graphic  picture  has 
boundaries,  by  and  which  the  user  can  not  proceed  beyond 
it.  A  two  dimensional  graphic  can  be  zoomed  in  or  out,  but 
the  information  on  the  object  remains  essentially  the  same. 

Three  Dimensional  Graphics 

Three  dimensional  graphics  involve  adding  a  third 
dimension  to  the  picture,  the  Z  axis.  It  is  important  to 
understand  that  all  computer  images  produced  on  a  graphics 
screen  are  always  two  dimensional;  thus  when  three 
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dimensional  graphics  are  mentioned,  the  term  refers  to  the 
underlying  model  used  to  generate  a  picture,  not  the 
picture  itself  (15:30).  There  are,  however,  new 
technologies  arriving  in  which  the  display  of  three 
dimensions  will  be  possible  (i.e.,  use  of  special  optics 
glasses).  This  is  somewhat  different  from  holographic 
images,  but  along  the  same  guidelines.  The  computer  stores 
in  its  memory  the  three  dimensional  data  points  or 
vertices,  of  the  object  being  rendered  into  a  two 
dimensional  picture.  From  these  points  polygons  are 
generated  producing  surfaces  of  different  colors  specified 
by  the  user.  Various  textures  can  also  be  achieved  on  these 
polygons,  such  as  a  mirror  like  texture,  glass,  or  metal. 
When  creating  a  three  dimensional  graphic,  the  user  must 
include  all  possible  information  on  the  object  to  insure 
accuracy  of  the  object  being  presented.  When  rendering  the 
graphic  in  three  dimensions,  the  computer  will  also 
generate  the  aliasing  effects  of  the  object  as  seen  from  a 
particular  view.  These  effects  are  the  result  of  a  three 
dimensional  light  source  inserted  by  the  user  at  a  given 
three  dimensional  coordinate.  Multiple  light  sources  can 
also  be  given  for  added  illumination  to  the  different  sides 
of  an  object  for  depth  perception. 

The  primary  benefits  of  three-dimensional  graphics  is 
the  amount  of  information  that  can  be  obtained  from  the 
picture.  More  than  one  side  of  an  object  can  be  displayed. 


The  object  can  be  zoomed  in  or  out  to  see  if  there  is  more 
information  involved  with  it.  Another  benefit  is  the 
realistic  and  photo-like  quality  which  a  three  dimensional 
graphic  presents  through  ray  tracing.  Ray  tracing  is  a 
technique  used  for  generating  shaded  images  on  the  object 
being  created  (29:96)  .  Ray  traced  three-dimensional 
objects  add  an  even  higher  resolution  than  two-dimensional 
graphics,  because  it  produces  images  in  a  high  resolution 
mode  of  over  4096  colors.  This  powerful  technique  produces 
astounding  images  that  no  picture  created  in  two 
dimensional  could  produce. 

One  major  disadvantage  of  three-dimensional  graphics 
is  the  amount  of  disk  storage  required.  In  addition  to  the 
disk  space  needed  to  store  the  picture  or  animation 
rendered  from  three  dimensions,  the  computer  must  also 
maintain  the  object  or  objects  in  question  on  disk  storage. 
Furthermore,  it  is  also  required  that  a  computer  have 
available  a  large  amount  of  memory  in  random  access  memory 
(RAM)  to  produce  the  three  dimensional  graphic.  The  three 
dimensional  graphic  image  generated  in  this  thesis  required 
five  megabytes  of  RAM.  This  is  due  to  the  processing  of 
the  object  in  three  dimensions  and  what  its  appearance  will 
be  from  a  particular  reference  point.  One  other 
disadvantage  to  three-dimensional  graphics  is  the  time 
involved  to  generate  a  picture.  A  three-dimensional 
picture  can  take  from  one  minute  to  as  much  as  one  week  to 
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generate,  depending  on  speed  of  the  computer.  Some 
projects  may  require  several  weeks  to  be  generated, 
depending  on  the  microprocessor  speed.  An  organization 
will  not  normally  wait  this  long  for  graphic  results  to 
assist  in  the  decision-making  process.  But  as  mentioned 
before  with  the  technology  available,  an  organization  can, 
provided  it  has  a  sufficient  budget,  purchase  a  fast  enough 
microprocessor  to  produce  the  graphics  in  minutes.  With 
these  two  different  types  of  computer  graphics  having  now 
been  explained  in  detail,  it  is  appropriate  to  discuss  some 
current  applications. 

Computer  Graphics  Applications 

As  mentioned  before  in  the  first  chapter,  there  are 
many  applications  for  either  two  or  three  dimensional 
computer  graphics.  Dr.  Tony  Diment,  a  graphics  expert,  has 
said:  'There  now  seems  to  be  no  area  of  human  endeavor 

that  cannot  be  enhanced  by  the  application  of  computer 
graphics'  (19:8).  One  example  is  the  applicability  of  real 
time,  three  dimensional  images  for  mass  market  products. 

In  an  article  written  by  Plummer  (23:143-150)  ,  a  research 
prototype  of  a  kitchen  designer  is  presented.  The  article 
compares  the  differences  between  using  a  two  dimensional  or 
three  dimensional  computer  graphics  system.  Plummer 
explains  the  differences: 
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Real  time  three  dimensional  graphics  allows  much 
greater  interaction.  A  conventional  two  dimensional 
system  will  allow  images  to  be  presented  on  the  screen 
and  choices  made.  A  three  dimensional  system  allows 
exploration  in  a  manner  which  does  not  have  to  be 
precisely  anticipated  by  the  designer  of  the 
application  package.  Furthermore,  the  world  being 
explored  can  be  imaginary  and  its  structure  may  have 
been  influenced  by  the  user.  (23:143) 

Another  application  of  computer  graphics  is  with 

simulations.  One  such  simulation  is  the  Advanced  Battle 

Simulation  (ABS)  used  for  Command  and  Control  practice  with 

the  U.S.  Army  (4:51-55).  ABS  is  an  extension  of  an  even 

more  elaborate  simulation  Network  (SIMNET)  Defense  Advanced 

Research  Projects  Agency  (DARPA)  -  Army  project 

(24:60-64) .  SIMNET  and  ABS  utilize  both  two  and  three 

dimensional  graphics,  with  each  playing  an  important  part 

to  ensure  the  user  gets  only  the  pertinent  information. 

Computer  graphics  for  training  individuals  has  strong 

potential.  One  application  in  training  is  the  use  of  three 

dimensional  graphics  in  air  weapons  controlling  of  three 

dimensional  graphics  in  air  weapons  controlling 

(11:1-37).  The  research  used  a  simulated  three  dimensional 

display  system  as  a  supplemental  training  guide  to  benefit 

air  weapons  controllers. 

Another  application  which  is  particularly  Important 
for  decision  support  systems,  is  one  created  by  a  thesis 
student  involving  adaptive  design  (12:1-165).  Gfeller 
designed  a  computer  based  decision  support  system  for 
evaluating  a  post-attack  chemical  contamination  of  an  air 
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base.  This  system  utilized  two  dimensional  computer 
graphics  to  aid  the  decision  making  process. 

From  the  two  types  of  computer  graphics  explained  in 
this  research  paper,  it  is  important  for  the  designer  to 
understand  what  is  the  organization’s  ultimate  objective. 
Both  types  have  their  advantages  and  disadvantages 
dependent  cn  the  type  of  information  being  presented. 
Computer  hardware  is  also  an  important  f  .ctor  to  take  into 
account  when  deciding  which  graphic  presentation  is 
feasible.  Production  cost  is  also  important.  Each  of  these 
factors  must  be  examined  carefully,  and  research  is  needed 
to  determine  in  greater  detail  the  differences  and  possible 
measurements  for  determining  what  can  be  the  best  choice. 

As  computer  graphics  technology  continues  to  improve  and 
become  cheaper,  faster  and  more  efficient,  the  applications 
using  computer  graphics  will  widen  even  further.  An 
understanding  of  why  a  specific  display  is  beneficial  for 
an  organization  or  decision  support  system  will  assist  in 
determining  where  and  what  type  can  be  used  in  other 
arenas.  Using  the  ABO  environment  is  one  such  example. 

Air  Base  Operability  is  an  excellent  environment  to 
demonstrate  the  use  of  computer  graphics  displays  because 
of  its  mission.  The  next  section  will  describe  how  ABO 
operates  and  how  it  can  utilize  computer  graphics 
animation . 
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Air  Base  Operability 


The  ABO  mission  is  to  provide  Air  Force  commanders 
information  concerning  the  capability  to  survive,  recover 
and  operate  under  attack  conditions  (5:Al-2).  One  of  the 
key  objectives  is  to  rapidly  recover  from  the  effects  of  an 
enemy  attack  on  the  air  base.  The  Survival  Recovery 
Center's  (SRC)  primary  function  in  the  ABO  is  to  advise  the 
wing  commander  and  higher  headquarters  on  the  status  of  the 
air  bases'  facilities.  In  order  to  do  this,  the  SRC  plots 
attack  damage  on  a  map,  transmits  the  direction  for 
recovery  efforts  and  recommends  priorities  for  recovery. 

The  ABO’S  recovery  mission  is  to  assess  the  damaged  areas, 
reduce  the  hazards  found  (unexploded  ordinance;  nuclear, 
biological,  and  chemical  contamination;  fire),  repair  the 
damage  to  critical  resources,  and  return  the  air  base  to 
maximum  combat  status  in  the  minimum  amount  of  time. 

Organization  of  ABO 

All  Air  Base  Operability  (ABO)  operations  are 
controlled  by  the  wing  commander  or  equivalent.  Assigned 
forces  operate  through  the  Wing  Operations  Center  (WOO 
using  the  battle  staff  and  existing  telephone  and  radio. 

The  combat  support  group  commander  is  responsible  for 
directing  the  activities  in  the  SRC.  He  also  'coordinates 
the  efforts  of  the  supporting  staff  to  analyze,  collect, 
prioritize,  display,  and  report  information  on  the  status 
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of  the  base'  (5:A2-8).  The  SRC  works  under  the  Wing 
Operations  Center  and  is  the  center  for  marking  and 
identifying  all  base  damage.  After  all  damage  has  been 
located,  the  SPC  is  then  responsible  for  devising  a 
recovery  strategy.  The  WOC  then  approves  this  plan  and 
implements  the  recovery  activity.  It  is  the  SRCs 
responsibility  to  determine  the  extent  of  the  damage  to 
facilities,  keeping  track  of  personnel  status,  resources, 
etc.  From  the  SRC,  the  Base  Civil  Engineer  (BCE)  will  send 
out  damage  assessment  teams  to  determine  the  extent  of 
damage  (6:2-7) .  He  will  also  receive,  review  and  evaluate 
damage  assessment  reports.  The  BCE  will  also  assist  the 
commander  in  building  and  achieving  the  base  recovery 
strategy . 

Damage  Assessment 

The  ABO  team  obtains  a  damage  assessment  immediately 
after  an  attack  has  occurred.  Damage  assessment  consists 
of  two  types  -  Rapid  Runway  Repair  (RRR)  and  facility 
damage  (6:3-1).  For  purposes  of  this  thesis,  only  facility 
damage  will  be  covered.  Facility  damage  involves  the 
evaluation  of  damage  caused  to  buildings,  POL  sites,  and 
other  utilities. 

There  are  two  types  of  information  -  location  of 
pavement  damage  caused  by  bombs,  cannon  fire,  etc.,  and 
locations  of  UXOs  which  are  transmitted  to  the  SRC.  All 
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UXOs  must  be  marked  within  300  feet  of  buildings  or 
aircraft  operating  surfaces.  Camouflet  craters,  very  deep 
craters  with  no  apparent  rupture  to  the  surface  of  a  road, 
must  be  reported.  The  maps  presented  must  be  scaled  to 
show  the  300  foot  UXO  radius-of -ef f ect  around  buildings. 

The  damage  must  be  recorded  once  it  is  evaluated,  then 
immediately  reported  to  the  SRC  for  damage  plotting.  The 
complete  understanding  of  the  information  being  relayed  is 
important  to  the  speed  of  reporting  and  strict  adherence  to 
radio  discipline  by  SRC  and  Damage  Assessment  Team  (DAT) 
personnel.  The  areas  to  be  evaluated  according  to  their 
importance  are  'the  takeoff  and  landing  (TOL)  surfaces; 
access  routes;  maintenance,  reloading,  refueling  areas;  and 
other  areas  directed  by  the  SRC  (6:3-11).  In  each  of 
these  areas,  all  craters  and  unexploded  ordinances  must  be 
included  in  the  report. 

Damage  Assessment  Information 

The  following  materials  are  used  for  damage  assessment 
inside  the  Survival  Recovery  Center  (SRC) : 

Base  Map  (Scale  1:4800) 

This  is  a  base  layout  map  to  indicate  the  location 
of  UXOs,  Damage  and  contamination. 

Airfield  map  (Scale  1:1200) 

Clear  plastic  crater  template  (Scale  1:1200) 

Colored  pencils. 

MOS  windows  -  clear  plastic  rectangles  representing 
MOS  length  and  width  (Scale  1:1200) 
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Chart  Selection  Tables 

Depicting  status  of  equipment,  resources  and 
personnel . 

Surface  roughness  charts  for  determining  MOS  size 
and  crater  repair  quality. 

Chart  holding  box 

EOD  time  and  bomb  damage  repair  worksheets 

(6:4-12) 

The  damage  assessment  in  the  SRC  is  conducted  in  4 
phases.  However  the  last  two.  evaluation  and 
recommendation  to  commander,  will  only  be  discussed  as  it 
is  only  pertinent  to  this  research  dealing  with  the 
post-attack  recovery  phase.  The  evaluation  phase  consists 
of  assessing  a  damaged  areas  to  determine  the  worst  impact 
and  selecting  MOS  (Minimum  Operating  Surfaces)  for  aircraft 
to  operate.  Damage  to  buildings  must  be  evaluated  in  case 
of  evacuation  if  necessary.  The  final  phase  is  reporting 
to  the  wing  commander  to  inform  him  so  that  implementation 
of  repairs  or  evacuation  can  be  started  as  soon  as 
possible.  These  last  two  phases  are  directly  involved  in 
the  decision  making  process.  These  two  phases  will  be 
initiated  when  conducting  the  research  to  determine  which 
computer  graphic  is  more  beneficial. 

Future  Airfield  Damage  Assessment  System 

There  are  two  systems  being  developed  for  the  ABO 
decision  making  area.  The  first  is  the  airfield  damage 
assessment  system  (ADAS)  and  the  second  is  named  CRISIS. 

The  ADAS  is  composed  of  an  airborne  subsystem  and  a  ground 
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assessment  subsystem  (6:3-15).  Developmental  testing 
occurred  during  the  'Salty  Demo'  (April-May  1985).  When 
the  system  becomes  fully  automated,  it  will  lead  to  a 
computer  aided  MOS  (Minimum  Operating  Surface)  selection 
process  for  determining  Bapid  Bunway  Bepair  (BBB) .  The 
CBISIS  system  was  developed  at  the  Air  Force  Academy 
(12:8).  It  serves  a  similar  purpose  for  the  SBC  (Survival 
Becovery  Center)  by  determining  damage  assessment,  chemical 
contamination,  etc.  Both  these  systems  demonstrate  the 
practical  application  of  two  or  three  dimensional  animation 
graphics  to  the  decision  making  area. 

Conclusion 

It  has  been  determined  that  the  use  of  either  two  or 
three  dimensional  computer  animation  graphics  into  the 
decision  making  area  of  ABO  is  practical.  The  next  chapter 
will  discuss  the  methods  for  comparing  both  types  of 
graphics  in  the  ABO  environment. 
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Ill . 


Methodology 


Overview 

This  chapter  describes  the  steps  which  will  be  taken 
to  answer  the  investigative  questions  given  in  chapter  I. 

It  will  also  thoroughly  describe  the  procedures  taken  to 
conduct  the  experiment  to  determine  the  preference  between 
two  or  three  dimensional  computer  graphics.  Thus  answering 
investigative  questions  1  and  2.  The  specific  objective 
will  be  to  determine  if  there  is  a  significant  difference 
between  the  different  computer  animation  graphics  displays 
and  which  will  be  more  beneficial  for  the  Air  Base 
Operability  decision  making  process.  The  specific  method 
chosen  for  this  research  was  the  true  experimental  design 
for  comparing  three  different  types  of  ABO  displays. 

Decision  Support  Systems 

In  order  to  conduct  and  compare  the  two  and  three 
dimensional  computer  animation  graphics  displays  in  a 
decision  making  environment,  as  mentioned  in  chapter  II,  a 
feasible  application  such  as  Air  Base  Operability  was 
necessary.  It  was  also  necessary  to  design  a  decision 
support  system  based  on  the  ABO  process  using  both  types  of 
graphic  displays,  because  ABO  is  planning  on  utilizing  a 
system  such  as  CRISIS  (12:5)  for  their  decision  making. 
Therefore,  in  order  to  compare  the  preferences  in  decision 
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making  ability,  this  researcher  designed  a  two  and  three 
dimensional  DSS  (Decision  Support  System) .  These  DSSs  are 
designed  to  meet  the  mission  requirements  of  ABO  -  - 
locating  facility  damage  during  the  post-attack  phase  of 
base  recovery.  These  two  systems  also  allow  the  user  to 
scan  through  the  simulated  air  base  and  locate  the  damaged 
areas,  thereby  making  decisions  whether  a  building,  POL 
depot,  or  runway  needs  to  be  evacuated,  repaired,  or 
rerouted.  Although  many  of  the  items  included  on  the 
storyboard  was  not  be  used  for  this  experiment,  it  was 
necessary  to  include  for  purposes  of  demonstrating  the 
importance  of  how  a  decision  support  system  operates. 

Evaluation  Importance 

Evaluation  of  the  ABO  DSS  is  necessary  to  be  certain 
that  it  will  determine  the  preference  of  between  the  two  or 
three  dimensional  computer  graphics.  Again  the  specific 
method  suggested  to  evaluate  the  ABO  decision  support 
system  will  be  to  conduct  a  true  experiment.  The  research 
method  would  involve  testing  a  group  of  individuals  exposed 
to  the  proposed  ABO  graphics  decision  support  system  or  the 
existing  ABO  decision  method  currently  using  maps,  charts 
and  tables.  Each  subject  would  have  hands  on  exposure  to 
one  or  the  other  systems.  The  current  ABO  decision  method 
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would  act  as  a  control  mechanism  to  determine  if  the  ABO 
decision  support  system  really  has  an  impact  to  the 
decision  making  process. 


Evaluation  Criteria 


It  is  important  to  evaluate  the  ABO  decision  support 
system  based  on  how  specifically  related  to  the  operational 
environment  for  which  it  is  intended  as  well  as  how  well 
the  users  cognitive  skills  are  enhanced  by  the  DBS’s 
graphical  interface.  The  following  operational  evaluation 
items  are  considered  for  testing  (some  suggested  by 
researcher  Andrew  Sage)  based  on  the  storyboard  and  concept 
map  already  designed: 

1.  Is  the  ABO  decision  support  system  logically  sound 
and  stable?  (25:88) 

2.  Is  the  ABO  DSS  transparent  to  the  user?  Is  it 
simple  to  use  and  idiot  proof?  Does  it  have  an  easy 
to  understand  help  reference  system? 

3.  Is  the  ABO  Decision  support  system  intuitive? 

Does  it  present  the  user  with  the  right  information 
when  and  where  he/she  needs  at  the  right  time? 

4.  Are  there  minimal  keystrokes  in  the  ABO  DSS?  Is 
the  graphics  interface  to  the  control  screen  simple, 
fast  and  quick  to  run  for  user  commands? 

5.  Is  the  DSS  closely  matched  up  to  the  operational 
and  organizational  situation  environment  of  the  actual 
Air  Base  Operability  (ABO)  decision  making  process? 
(25:88) 

6.  Does  the  ABO  DSS  support  the  variety  of  cognitive 
skills,  styles  and  knowledge  to  the  user  operating  the 
system?  (25:88) 
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7.  Does  the  ABO  DSS  assist  the  user  with  the  aid  of 
developing  and  using  their  own  cognitive  skills, 
styles  and  knowledge?  (25:86) 

8.  Is  the  ABO  DSS  sufficiently  flexible  for  allowing 
adaptability  in  use  for  users  with  differing  cognitive 
skills,  styles  and  knowledge?  (25:88) 

9.  Does  the  ABO  DSS  encourage  users  to  bring  in 
specific  Job  experiences  into  the  system  to  allow  more 
effective  solutions  to  unstructured  problems  and  will 
they  still  meet  the  political  and  institutional 
acceptability  constraints  of  the  ABO  organization? 
(25:88) 

10.  Will  the  ABO  DSS  promote  effective  long  term 
management,  i.e.  future  organizational  goals,  mission 
statements,  procedures?  (25:88) 


Other  Objectives  for  Evaluation 

Besides  identifying  the  criteria  to  determine  if  the 
ABO  decision  support  system  is  successful  or  not,  it  is 
appropriate  to  look  at  other  objectives  as  well.  These 
other  objectives  are  such  things  as  determining  the  most 
useful  strategy  for  employment  of  the  ABO  DSS  when 
conducting  the  experiment. 

Questioning  the  subjects  after  the  experiment  has  been 
conducted  may  be  appropriate.  The  users  could  be  asked 
about  their  conceptions  to  the  graphical  interface  of  the 
ABO  decision  support  system.  The  questionnaire  could  also 
find  out  the  behavioral  or  human  factor  effectiveness  of 
the  ABO  DSS.  It  is  also  important  to  make  sure  there  is  an 
appropriate  balance  between  the  Air  Base  Operability 
decision  support  system  evaluation  and  the  control  group 
using  the  current  ABO  decision  process  during  the 
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experiment.  Evaluation  of  the  ABO  DSS  using  the  four  Ps 
(Productivity,  Process,  Perception,  Product)  is  important 
also. 

In  examining  the  productivity  of  the  DSS  one  should 
ask  -  How  productive  is  the  ABO  DSS  to  accomplishing  the 
specific  task  of  identifying  base  facility  damage?  The 
process  of  making  the  decision  is  important  -  Does  the 
decision  making  process  change  at  all  with  the  DSS  as 
compared  to  the  existing  methods  used?  This  criteria  has 
already  been  mentioned  before  with  some  of  the  above 
criteria.  The  perceptions  of  the  decision  maker  when 
he/she  uses  the  ABO  decision  support  system  is  important. 
Again  this  might  be  determined  with  the  help  of  questioning 
the  user  after  he  has  been  exposed  to  the  ABO  DSS.  And 
finally  the  product  of  the  ABO  DSS  -  Is  it  cost  effective 
to  use  this  DSS  over  the  existing  system?  Will  the  DSS  be 
capable  of  running  on  existing  systems  used  by  Air  Base 
Operability?  All  of  these  other  objectives  could  help  even 
more  to  evaluate  the  ABO  decision  support  system. 

Evaluation  Criteria  Effects  on  Storyboard 

The  evaluation  criteria  could  possibly  change  the 
structure  and  appearance  of  the  preliminary  storyboard 
designed  for  the  ABO  decision  support  system.  It  is 
doubtful  however  that  the  evaluation  criteria  will  affect 
the  concept  map  of  the  ABO  DSS.  This  is  because  the 
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concept  map  was  designed  based  on  existing  procedures  in 
the  Air  Base  Operability  organization.  These  will  only 
change  if  the  ABO  organization  drastically  changes  its 
mission  statement  or  reporting  procedures. 

The  evaluation  criteria  might  change  the  storyboard  in 
some  way.  If  after  conducting  the  experiment,  the  users 
might  use  one  of  the  display  presentations  more  than  the 
other,  i.e,  using  the  three  dimensional  graphic 
presentation  over  the  two  dimensional  graphic  presentation 
of  facility  damage,  this  would  result  in  taking  out  that 
portion  of  the  decision  support  system.  If  one  animation 
graphic  is  more  preferable  to  the  user  then  both  should  not 
be  included  into  the  ABO  DSS. 

Some  other  changes  to  the  storyboard  might  be  the 
explanation  of  some  of  the  display  screens.  Since  these 
are  to  be  included  in  the  help  menu  section  of  the  DSS,  if 
the  user  has  difficulty  in  understanding  how  a  particular 
menu  function  works,  then  better  explanation  is  needed  to 
solve  that  misconception  or  misunderstanding. 

Overall  this  researcher  believes  there  will  be  very 
little  changes  needed  to  the  storyboard.  Although  it  is 
somewhat  certain  that  the  deletion  of  either  the  two  or 
three  dimensional  animation  display  will  occur  since  the 
primary  reason  for  this  ABO  decision  support  system  is  to 
determine  which  display  is  more  preferable. 
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Variables  of  Measurement 


In  order  to  determine  the  preference  between  the 
different  ABO  displays,  it  is  necessary  to  identify  the 
measurement  variables.  These  are  dependent  variables  which 
will  be  used  in  determining  a  statistical  significant 
difference.  Here  are  some  areas  to  consider: 

1.  The  average  time  it  took,  each  group  to  assess  the 
damage  in  the  model . 

2.  How  well  each  subject  assessed  the  damage  in  the 
display  (i.e.  number  of  damaged  facilities). 

3.  How  well  each  subject  correctly  assessed  the 
seriousness  of  the  damaged  facilities. 

4.  Any  behavioral  traits  that  might  arise  when  the 
subject  is  using  the  ABO  decision  support  system. 

Therefore  based  on  the  above  items,  the  following 

variables  of  measurement  are  created  to  be  tested  during 

the  experiment: 

Variable  1  -  DAMAGE  TYPE:  This  variable  will  consist  of 
the  type  of  damage  done  to  a  facility.  It  will  measure  the 
accuracy  of  how  the  subject  correctly  identified  the  type 
of  damage.  The  different  types  of  damage  shown  on  the 
displays  are:  Craters,  Cracks  and  UXOs . 

Variable  2  -  SEVERITY:  This  variable  consists  of  the  type 
of  seriousness  of  the  damage.  It  will  measure  how  well  the 
subject  correctly  assessed  the  damaged  area  and  identified 
its  severity.  This  variable  also  measures  how  well  the 
subject  identifies  it  by  color  coding.  Each  type  of 
severity  is  color  codec,  as  follows: 
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MINIIIAL 


BLACK 


MEDIUM  -  DABK  RED 
CRITICAL  -  BRIGHT  RED 

Variable  3  -  LOCATION;  This  variable  consists  of  the 
location  of  the  damaged  area.  It  will  measure  how 
accurately  the  subject  is  able  to  pinpoint  a  damaged  area’s 
true  location  based  on  what  the  subject  sees  on  the  graphic 
display.  The  location  choices  in  the  experiment  are: 

North,  South,  East,  West  and  Top  (for  Buildings). 

Variable  4  -  AREA:  This  variable  consists  of  the  type  of 
location  the  damaged  area  lies  on.  It  will  not  only 
measure  again  how  accurately  the  subject  is  able  to 
determine  the  location  of  the  damaged  area,  but  how  well 
the  subject  can  see  the  damaged  area  on  a  particular 
graphic  object.  The  area  choices  available  are:  Wall, 
Ground  and  Roof . 

All  the  variables  of  measurement  were  compiled 
together  in  a  questionnaire  sheet  (See  Appendix  B) .  This 
questionnaire  explains  to  the  subject  the  mission  objective 
of  the  ABO  scenario  and  the  variables  he/she  must  look  for. 
The  questionnaire  also  gives  an  example. 

Data  Collection  Procedures 

The  variables  of  measurement  to  be  analyzed  in  this 
research  experiment  will  be  collected  from  each  of  the 
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experiment  groups.  Each  subject  will  mark  their  answers  on 
a  computerized  answer  sheet  from  the  above  mentioned 
questionnaire.  Next,  the  answers  will  be  read  into  a 
computer  for  statistical  analysis.  There  must  be  at  least 
thirty  subjects  tested  all  together.  This  meets  the 
criteria  for  the  Central  Limit  Theorem  in  statistics  that 
thirty  sample  data  points  will  be  sufficient  to  represent 
the  group  population  (9:291).  Each  subject  will  be 
randomly  chosen  to  utilize  one  of  the  three  DSS  displays. 
The  two  and  three  dimensional  DSS  displays  will  be 
presented  through  an  Amiga  2000  computer  system.  This 
system  was  chosen  by  the  researcher  because  of  his 
familiarity  with  designing  computer  animation  graphics  on 
the  Amiga.  Each  subject  will  look  at  the  display  and  given 
a  maximum  time  of  fifteen  minutes,  determine  the  amount  of 
damage  the  simulated  air  base  has  been  attacked.  Each 
subject  will  record  the  amount  of  damage  that  has  occurred 
to  the  air  base  by  filling  out  a  computerized  answer  sheet. 
This  sheet  contains  all  buildings,  POLs,  and  runways 
necessary  for  the  damage  recording.  It  also  contains  the 
spaces  available  for  type  of  damage,  severity  of  the 
damage,  location  and  area  as  seen  on  the  DSS  display. 

Specification  of  Variables 

An  attempt  will  be  made  to  control  the  variables  in 
the  experiment  to  assure  external  validity.  Each  subject 
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will  receive  the  same  instructions  for  completing  the 
specified  mission  task.  Each  of  the  variables  for  each 
group  are  exactly  the  same.  It  will  be  appropriate  to 
determine  whether  these  variables  might  affect  other 
variables  in  some  way.  The  group  tested  by  the  standard 
ABO  display  will  serve  as  a  control  group  against  the  other 
two  groups  of  experimental  displays.  The  exar.ination 
occurs  during  the  post- test  phase  of  the  experiment.  There 
will  be  no  pretest  on  the  subjects  due  to  using  the 
standard  ABO  display  as  a  control  group. 

Experimental  Design  and  Conditions 

As  mentioned  above,  the  experiment  will  consist  of 
three  different  types  of  DSS  displays,  the  standard  display 
currently  used  by  ABO,  the  two  dimensional  display,  and  the 
three  dimensional  display.  All  three  displays  will  hold 
the  same  damage  data  in  order  to  assure  internal  validity 
for  all  the  Decision  Support  Systems.  Internal  validity  is 
threatened  if  there  are  any  changes  between  the 
observations  or  measuring  instruments.  Therefore,  all 
model  displays  will  represent  the  same  simulated  air  base. 
All  models  will  have  exactly  the  same  predefined  damaged 
areas  to  facilities  for  the  subjects  to  locate  and 
identify.  After  the  DSS  displays  were  designed,  they  were 
examined  by  local  experts  who  were  familiar  with  computer 
graphics,  decision  support  systems,  or  Air  Base  Operability 
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procedures.  These  experts  were  Lt  Col  Dorthy  McBride,  Lt 
Col  John  Valusek  and  Capt  Polasek.  All  the  experts  were 
satisfied  that  these  DSS  displays  were  sufficient  to  use 
for  the  experiment.  After  the  displays  were  examined  the 
next  step  was  to  add  damage  to  the  simulated  air  base. 
Following  this  a  trial  test  was  conducted  again  to  ensure 
each  DSS  display  would  operate  correctly  for  each  subject 
while  tested.  The  following  briefly  describes  each  DSS 
display . 

Simulated  Air  Base 

First,  each  DSS  display  had  to  be  constructed  from  a 
single  air  base.  This  simulated  air  base  was  built  in 
three  dimensions.  Three  dimensional  models  can  be 
constructed  in  many  ways,  through  digitizing,  data  base 
input  or  Computer  Aided  Design  (CAD) ,  as  mentioned  before 
in  Chapter  II.  The  experimental  air  base  this  researcher 
built  was  constructed  with  the  hel.  of  a  CAD  program  called 
SCULPT  4D.  The  dimensions  of  each  building,  runway  and  POL 
facility  were  constructed  with  x,  y  and  z  coordinates  ie, 
length,  width  and  height.  After  the  air  base  was  fully 
constructed,  objects  representing  craters,  UXOs  and 
building  cracks  were  added.  These  objects  wer'  also  in 
different  colors  representing  the  extent  of  the  damage  ie, 
bright  red  represents  a  critical  damage  point  and  the 
facility  must  be  evacuated.  The  color  coding  will  be 
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explained  in  detail  later.  The  entire  air  base  model 
including  the  damaged  objects  consisted  of  8,000  points  and 
21,000  polygons.  This  required  over  3.5  megabytes  of  RAM 
(Random  Access  Memory)  for  the  Amiga  computer  to  hold  the 
entire  air  base  in  memory  and  generate  the  required 
computer  graphic  pictures. 

ABO  Standard  Display 

This  display  is  based  on  the  standard  display 
currently  used  by  Air  Base  Operability.  The  SRC  mainly 
charts  their  damage  area  information  on  a  large  blueprint 
of  the  air  base.  As  mentioned  in  Chapter  II,  they  receive 
reports  from  damage  assessment  teams  (DATs)  and  then  record 
them  on  this  air  base  blueprint.  Therefore  it  was 
necessary  to  construct  a  similar  blueprint  of  the  simulated 
air  base  from  t.  is  experiment.  A  top  down  view  was 
generated  from  the  Amiga  computer.  This  picture,  after 
constructed,  was  a  wire  frame  only  and  showed  no  damaged 
areas.  The  graphic  picture  display  dimensions  were  2500  X 
2500  pixels  in  width.  This  was  necessary  in  order  to 
create  a  blueprint  paper  hardcopy  of  equivalent  size 
compared  to  the  standard  ABO  map  used. 

After  the  blueprint  was  constructed,  it  was  necessary 
to  create  a  DAT  report  document  for  the  subject  to  examine 
for  recording  the  damage  onto  the  blueprint.  This  report 
listed  all  the  damaged  areas  that  occurred  during  the 
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attack  on  the  simulated  air  base.  Nothing  was  omitted  for 
the  purpose  of  validity.  The  damage  entries  on  the  report 
were  placed  in  random  order  and  the  subject  was  to  assume 
they  were  listed  in  order  of  time  reported  (See  Appendix 
C)  . 

The  procedure  for  reporting  damage  in  this  display  was 
to  examine  the  DAT  report,  accurately  record  a  damaged  area 
onto  the  air  base  blueprint  and  then  record  the  damaged 
area  onto  the  computerized  answer  sheet.  In  order  for  a 
successful  recorded  damaged  area  to  count,  it  must  be 
recorded  on  both  the  air  base  blueprint  and  the  answer 
sheet.  This  approximately  estimated  the  time  it  takes  for 
an  SRC  representative  to  decode  an  incoming  DAT  report  and 
then  record  it  onto  their  air  base  map. 

Two  Dimensional  Display 

The  two  dimensional  display  represented  a  top  down 
view  of  the  simulated  air  base.  This  measured  the 
characteristics  of  two  dimensions  as  mentioned  in  Chapter 
II.  The  two  dimensional  decision  support  system 
constructed  begins  with  an  overall  top  down  view  of  the 
entire  air  base.  This  overall  view  was  generated  by 
looking  down  on  the  three  dimensional  air  base  model  at  a 
simulated  height  of  1000  feet.  The  view  was  generated 
after  the  air  base  was  damaged.  Therefore,  some  damaged 
areas  were  visible  and  the  subject  could  already  ascertain 
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some  answers  based  on  this  view.  From  here  the  subject  is 


then  able  to  get  a  closer  detailed  two  dimensional  look  at 
a  facility.  The  DSS  is  designed  to  show  a  zoomed  in  2-D 
picture  by  Just  selecting  (left  button  on  a  mouse 
controlled  input  device)  a  the  particular  facility.  After 
selecting  the  desired  facility,  the  two  dimensional  graphic 
picture  is  shown.  This  picture  was  again  generated  from 
the  three  dimensional  air  base  model  and  shows  a  top  down 
view  only  of  the  facility,  at  a  height  of  approximately  300 
feet.  After  viewing  the  zoomed  in  picture,  the  subject  can 
go  back  to  the  overall  view  by  simply  pressing  the  ’Esc’ 
button  on  the  computer  keyboard. 

The  procedure  for  this  display  is  for  the  subject  to 
accurately  find  and  record  the  damaged  areas  on  the  air 
base.  The  subject  is  also  informed  he/she  must  first 
locate  the  most  serious  damaged  areas  (based  on  color 
coding) .  If  the  subject  sees  a  damaged  area  from  the 
overall  top  down  view,  then  he/she  is  instructed  to  view 
the  closer  two  dimensional  view.  After  all  damaged  areas 
are  perceived  from  the  overall  view,  the  subject  is  then 
recommended  to  look  at  other  facilities  to  check  if  any 
damaged  areas  were  missed.  After  locating  a  damaged  area 
the  subject  is  to  correctly  record  it  on  the  computerized 
answer  sheet. 
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The  two  dimensional  graphic  display  contains  all  the 
damaged  areas.  Similar  to  the  standard  ABO  display  and  for 
the  purpose  of  validity,  no  damaged  areas  were  omitted. 

Three  Dimensional  Display 

Finally,  the  three  dimensional  ABO  display  consisted 
of  a  three  dimensional  view  of  the  simulated  air  base. 

Each  facility  had  its  own  computer  generated  graphic 
animation  view.  This  view  entailed  an  animated  view 
starting  from  the  top  view  of  the  facility,  moving  downward 
for  a  north  view,  then  rotated  clockwise  to  view  the 
facility  until  returning  back  to  the  north  view.  All  of 
the  3-D  animation  graphics  are  constructed  from  the  same 
simulated  air  base.  All  the  three  dimensional  views  of 
facilities  were  accessed  in  the  same  manner  as  the  two 
dimensional  display.  The  subject  from  the  beginning  saw  an 
overall  top  down  view  of  the  simulated  air  base.  In  order 
to  view  a  three  dimensional  animated  view  of  a  facility, 
the  subject  selected  that  facility,  by  moving  the  mouse 
and  clicking.  The  top  down  view  is  identical  to  the  two 
dimensional  top  down  view.  When  the  animation  was  viewed 
of  a  particular  facility,  the  subject  had  some  control  of 
how  the  animation  is  presented.  This  control  consisted  of 
the  ability  to  change  the  speed  of  the  animation;  pause  the 
animation  at  any  time;  or  restart  the  animation  back  to  the 
beginning . 
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The  procedure  for  this  display  was  exactly  the  same  as 
the  two  dimensional  display.  The  subject  was  to  identify 
the  most  seriously  damaged  areas,  first  picking  areas  from 
the  overall  top  down  view,  then  working  on  other  facilities 
not  so  obvious.  After  a  damaged  area  had  been  identified, 
the  subject  then  should  have  correctly  recorded  that 
information  on  the  computerized  answer  sheet. 

Explanation  of  the  Controls 

Each  display  group  consisted  of  a  random  number  of 
individual  subjects.  All  subjects  were  randomly  selected 
into  one  of  the  display  groups  as  they  arrived  for  testing. 
All  the  subjects  were  given  a  pretrial  briefing  to  explain 
the  mission  scenario  for  ABO,  the  objective  to  that 
mission,  and  procedures  for  recording  onto  the 
computerized  answer  sheet.  Each  subject  were  not  be 
exposed  to  any  other  display  other  than  the  one  they  were 
assigned  to. 

Criteria  for  Selection 

There  was  no  specific  criteria  for  selection  of  the 
subjects  for  the  experiment.  This  researcher  posted 
announcements  concerning  the  experiment  to  all  who  might  be 
interested  in  participating.  The  only  criteria  for 
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non-selection  was  be  if  the  subject  was  severely  color 
blind,  because  the  variables  mentioned  above  involved  color 
coding  for  severity  of  damage. 

Data  Analysis  Procedures 

Data  analysis  first  involved  associating  the  data 
statistically  to  ensure  normality  of  the  data.  All  three 
data  groups  were  analyzed  together  to  ensure  a  normal 
sample  size  of  the  population  and  distribution  has 
occurred.  This  was  done  using  a  univariate  analysis  of  the 
data.  It  was  then  determined  whether  the  appears  to  be  any 
statistical  relationship  between  the  answered  data  in  the 
three  display  groups.  Similarities  and  differences  within 
and  between  the  three  data  groups  were  detected  by  use  of 
the  parametric  statistical  procedure  known  as  Bonferroni’s 
multiple  comparison  procedure  for  analysis  of  variance 
(ANOVA) .  With  the  use  of  ANOVA,  several  assumptions  must 
be  made  and  followed  about  each  group  population  from  which 
the  data  is  gathered: 

1.  The  observations  must  be  independent. 

2.  The  populations  from  which  the  data  are  generated 
all  are  normally  distributed  with  standard  deviations 
and  means  which  may  or  may  not  be  unique. 

3.  The  level  of  the  data  obtained  from  the 
computerized  answer  sheets  will  be  at  least  interval. 

4.  The  data  is  gathered  in  a  random  and  independent 
process . 
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The  Bonferroni  procedure  will  effectively  compute  an 
interval  length  as  a  function  of  the  mean  square  error  of 
the  data.  Where  the  sample  means  differ  by  more  than  the 
value  of  the  computed  interval,  the  difference  in  the  true 
mean  is  stated  to  be  statistically  significant.  If  the 
sample  means  are  different  by  a  value  less  than  the 
computed  interval,  the  difference  will  be  considered 
significant  (21:665).  The  data  analysis  will  be  conducted 
through  the  use  of  the  computer  facilities  at  the  Air 
Force  Institute  of  Technology  (AFIT) ,  Wright  Patterson  Air 
Force  Base.  Ohio.  The  statistical  analysis  program  used 
for  testing  by  the  Bonferroni  procedure  is  called  the  SAS 
software  system.  SAS ’ s  PROC  ANOVA  procedure  will  be  used 
with  the  BON  (Bonferroni)  option.  This  will  be  used  for 
the  variance  of  means  computations. 

The  independent  variables  will  be  compared  from  the 
group  class  to  determine  significance.  The  damage, 
severity,  and  location  variables  will  be  compared  from  all 
three  groups.  If  there  is  a  statistical  significant 
difference  in  the  mean  scores  of  each  of  these  variables 
then  there  will  be  a  preferencial  difference. 

Reporting  Analysis  of  Data 

Chapter  IV  involves  the  implementation  of  the 
experiment  proposed  in  this  chapter  and  the  analysis  of  the 
data  collected.  Chapter  V  will  contain  the  summary  of  the 
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finding  and  conclusions  drawn  from  the  analysis.  These  two 
chapters  will  determine  wnether  the  proposed  hypothesis  is 
true  or  not  and  determine  the  preference  between  the  two  or 
three  dimensional  computer  graphic  displays. 
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IV.  Analysis  of  Data 


Overview 

The  previous  chapter  described  the  methodology  for 
conducting  the  true  experiment.  This  chapter  analyses  the 
data  compiled  from  the  true  experiment  to  determine  if 
there  is  a  significant  difference  between  the  three 
displays  --  three  dimensional,  two  dimensional,  and  the 
standard  ABO  Display. 

Collection  of  the  Data 

The  data  from  each  subject  was  collected  on  a 
computer  answer  sheet  as  mentioned  before  in  Chapter  III. 
After  all  subjects  were  tested,  the  data  was  then  compiled 
together  into  separate  groups  of  the  designated  variables. 

A  separate  variable  group  was  also  assigned  just  for 
interest  sake  which  identified  the  total  number  of 
responses  each  subject  made  whether  correct  or  not. 

Frequency  Analysis 

Each  variable  group  was  analyzed  by  frequency  and 
cumulative  frequency  to  compare  the  highest  and  most 
frequent  number  of  correct  answers  (See  Tables  1-12).  The 
variable  names  for  each  dependent  variable  were:  TOT  -  for 
Total  number  of  responses  the  subject  entered;  DAM  -  for 
total  number  of  correct  responses  the  subject  made  for  type 
of  damage  identified;  SEV  -  for  total  number  of  correct 
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responses  the  subject  made  for  type  of  severity  of  the 
damage:  and  LOG  -  for  total  number  of  correct  responses  the 
subject  made  for  the  location  of  the  damage  on  the  airbase. 
A  number  also  followed  the  variable  for  each  group  that 
variable  belonged  to:  3  for  three  dimensional  display;  2 
for  two  dimensional;  and  1  for  ABO  standard  display  map. 
There  were  14  subjects  tested  for  the  three  dimensional 
display,  11  subjects  for  the  two  dimensional  display,  and 
11  for  the  standard  ABO  display.  The  Mean,  standard 
deviation,  maximum  and  minimum  values  are  shown  for  all 
data  group  variables  together  in  table  1  below.  Appendix  C 
shows  a  frequency  listing  of  each  variable  group 
separately . 


Table  1  -  Mean  Data  of  a  Group  Variables 


Variable 

N 

Mean 

Std  Dev 

Minimum 

Maximum 

T0T3 

14 

16.43 

3.72 

8.00 

23.00 

T0T2 

1 1 

16.55 

3.11 

11.00 

21.00 

TOTl 

1 1 

12.09 

5.24 

3.00 

21.00 

DAM3 

14 

16.07 

3.34 

8.00 

20.00 

DAM2 

1 1 

13.55 

3.11 

10.00 

20.00 

DAMl 

1 1 

11.64 

5.05 

3.00 

20.00 

SEV3 

14 

10.57 

3.88 

2.00 

17.00 

SEV2 

1 1 

10.00 

3.49 

5.00 

16.00 

SEVl 

1 1 

9.27 

4.69 

2.00 

17.00 

L0C3 

14 

14.00 

4.30 

8.00 

21.00 

L0C2 

1 1 

13.55 

4.91 

3.00 

20.00 

LOCI 

1 1 

11.09 

5.03 

2.00 

20.00 

Some  of 

these 

frequencies  are 

close  bv:'- 

higher  than 

the  mean  of  each  variable.  It  was  also  important  to  view 


the  data  with  each  dependent  variable  group  combined  -- 
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that  is.  combine  all  the  data  together  and  compare  the 
distribution  by  each  dependent  variable  separately.  Table 
2  shows  the  Mean  data  of  the  dependent  variables  with  36 
data  points.  This  combined  group  data  allows  ANOVA 
comparisons  later  on  in  the  chapter.  The  group  dependent 
variables  named  are:  TOTAL  -  for  Total  responses;  DAMAGE  - 
for  correct  type  of  damage  identified;  SEVERITY  -  for 
correct  type  of  damage  severity  identified;  and  LOCATION  - 
for  correct  location  of  damage  identified. 


Table  2  -  Combined  Group  Mean  Data 


Variable 

N 

Mean 

Std  Dev 

Mini mum 

Maximum 

TOTAL 

36 

15.14 

4.47 

3.00 

23.00 

DAMAGE 

36 

13.94 

4.21 

3.00 

20.00 

SEVERITY 

36 

10.00 

3.96 

2.00 

17.00 

LOCATION 

36 

12.97 

4.75 

2.00 

21.00 

Frequency  Bar  Charts 

To  get  a  better  and  clearer  observation  of  how  evenly 
distributed  the  dependent  variable  groups  were  distributed, 
it  was  necessary  to  look  at  them  by  creating  frequency  bar 
charts.  Figures  5-16  (see  Appendix  D)  show  the 
distribution  for  each  dependent  variable  group.  It  became 
obvious  since  each  variable  group  had  a  small  sample  size 
that  the  bell  shape  of  the  bar  charts  was  not  apparent  in 
all.  In  fact  some  of  the  charts  were  skewed  to  the  left  or 
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right.  It  may  be  possible  that  if  more  data  was  added,  the 
bell  shape  would  be  more  apparent  thus  showing  an  even 
distribution . 

Again  it  is  important  to  view  the  data  grouped 
together  with  each  dependent  variable  viewed  separately. 
These  next  charts  (Figures  1-4)  show  a  more  evenly 
distributed  sample  size.  This  corresponds  to  the  Central 
Limit  Theorem  stated  in  Chapter  III. 
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Figure  2.  Bar  Chart  of  Correct  Dasiage 
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Figure  3.  Bar  Chart  of  Correct  Severity 


Figure  4.  Bar  Chart  of  Correct  Location 

Univariate  Procedure 

Next  it  is  necessary  to  view  each  dependent  group  of 
individual  data  for  normality.  By  conducting  a  univariate 
study  on  each  dependent  variable,  the  analysis  can  prove 
that  the  sample  groups  are  from  a  normal  population.  The 
univariate  test  performs  different  tests  for  normality 
depending  on  the  sample  size  of  the  data.  The  univariate 
procedure  consists  of  using  the  Wilk-Shapiro  test  for 
normality  for  sample  sizes  less  than  51.  The  following 
test  was  conducted  for  each  dependent  variable  group.  Each 
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figure  shows  a  different  dependent  variable  group  with  two 
types  of  output.  The  first  type  shows  descriptive 
statistics  for  calculating  the  Wilk-shapiro  normality  test. 
The  second  portion  contains  one  plot,  the  normality  plot. 
The  Normality  plot  shows  asterisks  («)  which  represent  the 
sample  data  and  plus  signs  (+)  which  represent  a  straight 
line.  if  the  sample  is  from  a  normal  distribution,  the 
asterisks  and  plus  signs  will  overlap.  A  large  number  of 
plus  signs  indicate  a  nonnormal  distribution  of  sample 
data.  By  rejecting  the  null  hypothesis  (Ho)  the  sample 
size  will  not  represent  a  normal  population  distribution. 
The  combination  of  these  two  indications  checked  the  data 
for  normality. 

In  figures  5-16,  a  univariate  procedure  was 
accomplished  on  each  of  the  data  groups.  Following  each 
procedure,  a  hypothesis  test  for  normality  was  conducted 
based  on  the  information  supplied  from  the  univariate  data. 
Conclusions  are  drawn  from  each  hypothesis  test  to 
determine  if  each  group  was  normally  distributed. 
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Variable=T0T3 
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Figure  5.  SAS  Univariate  Procedure  for  T0T3 


Test  for  Normality  on  T0T3 

The  data  consist  of  a  random  sample  XI ,  X2  . . .  ,  Xn 

of  size  n  =  14  for  the  T0T3  group. 

ASSUMPTIONS:  The  sample  is  a  random  sample. 

HYPOTHESIS: 


Ho:  Group  samples  are  from  a  normal  population. 
Ha:  Group  is  nonnormal. 

TEST  STATISTIC: 


TS:  W  =  0.961315 
alpha  =  .05 
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REJECTION  REGION: 


W  <  0.874 

CONCLUSION: 

There  is  insufficient  evidence  to  reject  the  null 
hypothesis  for  the  T0T3  data  group.  Therefore  the  sample 
population  is  normally  distributed  for  the  purposes  of  this 
experiment . 


Variable=T0T2 
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Figure  6.  SAS  Univariate  Procedure  for  TOT2 

Test  for  Normality  on  T0T2 


The  data  consist  of  a  random  sample  XI ,  X2  . . .  ,  Xn 

of  size  n  =  11  for  the  T0T2  group. 

ASSUMPTIONS:  The  sample  is  a  random  sample. 
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HYPOTHESIS: 


Ho:  Group  samples  are  from  a  normal  population. 
Ha:  Group  is  nonnormal. 

TEST  STATISTIC : 

TS:  W  =  0.935988 
alpha  =  .05 

REJECTION  REGION: 

W  <  0.850 

CONCLUSION: 

There  is  insufficient  evidence  to  reject  the  null 
hypothesis  for  the  T0T2  data  group.  Therefore  the  sample 
population  is  normally  distributed  for  the  purposes  of  this 
experiment . 
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Variable=TOT 1 
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Figure  7.  SAS  Univariate  Procedure  for  TOTl 

Test  for  Normality  on  TOT  1 

The  data  consist  of  a  random  sample  XI ,  X2  ...  ,  Xn 

of  size  n  =  11  for  the  TOTl  group. 

ASSUMPTIONS:  The  sample  is  a  random  sample. 

HYPOTHESIS: 


Ho:  Group  samples  are  from  a  normal  population 
Ha:  Group  is  nonnormal. 

TEST  STATISTIC: 


TS:  W  =  0.948855 
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alpha 


.05 


REJECTION  REGION: 

W  <  0.850 

CONCLUSION: 

There  is  insufficient  evidence  to  reject  the  null 
hypothesis  for  the  TOTl  data  group.  Therefore  the  sample 
population  is  normally  distributed  for  the  purposes  of  this 
experiment . 


Var iable=DAM3 


Moments 


N 

14 

Sum  Wgts 

14 

Mean 

16.07143 

Sum 

225 

Std  Dev 

3.338915 

Variance 

1  1  .  14835 

Skewness 

-1 .01274 

Kur tos i s 

1  .  130162 

USS 

3761 

CSS 

144.9286 

CV 

20 . 77547 

Std  Mean 

0.892363 

T : Mean=0 

18 . 00998 

Prob> !T! 

0.0001 

Num  *  =  0 

14 

Num  >  0 

14 

M(Sign) 

7 

Prob> ! M! 

0.0001 

Sgn  Rank 

52.5 

Prob> : s : 

0.0001 

W ;  N  o  r  ma  1 

0 . 906152 

Prob>  W 

0. 1374 

Normal  Probability  Plot 

21+  »++++« 

»«»+«+»++ 

«++»++ 

+ - + - ♦ - + - + - ♦ - + - + - + - + 

-2  -1  0  +1  +2 


Figure  8.  SAS  Univariate  Procedure  for  DANS 

Test  for  Normality  on  DAM3 

The  data  consist  of  a  random  sample  XI ,  X2  . . .  ,  Xn 
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of  size  n 


14  for  the  DAMS  group. 


ASSUMPTIOMS :  The  sample  is  a  random  sample. 

HYPOTHESIS: 

Ho:  Group  samples  are  from  a  normal  population. 
Ha:  Group  is  nonnormal. 

TEST  STATISTIC: 

TS:  W  =  0.906152 
alpha  =  .05 

REJECTION  REGION: 

W  <  0.874 

CONCLUSION: 

There  is  insufficient  evidence  to  reject  the  null 
hypothesis  for  the  DAMS  data  group.  Therefore  the  sample 
population  is  normally  distributed  for  the  purposes  of  this 
exper i men  t , 
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Variable=DAM2 
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Figure  0.  SAS  Univariate  Procedure  for  DA1I2 


Test  for  Normality  on  DAM2 

The  data  consist  of  a  random  sample  XI ,  X2  . . .  ,  Xn 

of  size  n  =  11  for  the  DAM2  group. 

ASSUMPTIONS:  The  sample  is  a  random  sample. 

HYPOTHESIS: 

Ho:  Group  samples  are  from  a  normal  population. 
Ha:  Group  is  nonnormal. 

TEST  STATISTIC: 

TS:  W  =  0.884731 
alpha  =  .05 
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REJECTION  REGION: 


W  <  0.850 

CONCLUSION: 

There  is  insufficient  evidence  to  reject  the  null 
hypothesis  for  the  DAM2  data  group.  Therefore  the  sample 
population  is  normally  distributed  for  the  purposes  of  this 
experiment . 
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Figure  10.  SAS  Univariate  Procedure  for  DAMl 
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Test  for  Normality  on  DAMl 


The  data  consist  of  a  random  sample  XI ,  X2  . . .  ,  Xn 

of  size  n  =  11  for  the  DAMl  group. 

ASSUMPTIONS:  The  sample  is  a  random  sample. 

HYPOTHESIS: 

Ho:  Group  samples  are  from  a  normal  population. 
Ha;  Group  is  nonnormal. 

TEST  STATISTIC: 

TS:  W  =  0.95886 
alpha  =  .05 

REJECTION  REGION: 

W  <  0.850 

CONCLUSION: 

There  is  insufficient  evidence  to  reject  the  null 
hypothesis  for  the  DAMl  data  group.  Therefore  the  sample 
population  is  normally  distributed  for  the  purposes  of  this 
exper i ment . 
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Variable=SEV3 
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Figure  11.  SAS  Univariate  Procedure  for  SEV3 

Test  for  Normality  on  SEV3 

The  data  consist  of  a  random  sample  XI ,  X2  . . .  ,  Xn 

of  size  n  =  14  for  the  SEV3  group. 

ASSUMPTIONS:  The  sample  is  a  random  sample. 

HYPOTHESIS: 

Ho;  Group  samples  are  from  a  normal  population. 
Ha:  Group  is  nonnormal. 

TEST  STATISTIC: 


TS:  W  =  0.967496 
alpha  =  .05 
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BEJECTION  BEGIOM: 


W  <  0.874 

CONCLUSION: 

There  is  insufficient  evidence  to  reject  the  null 
hypothesis  for  the  SEV3  data  group.  Therefore  the  sample 
population  is  normally  distributed  for  the  purposes  of  this 
experiment . 
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Figure  12.  SAS  Univariate  Procedure  for  SEV2 

Test  for  Normality  on  SEV2 

The  data  consist  of  a  random  sample  XI,  X2  ...  ,  Xn 

of  size  n  =  11  for  the  SEV2  group. 
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ASSUMPTIONS:  The  sample  is  a  random  sample. 

HYPOTHESIS: 

Ho:  Group  samples  are  from  a  normal  population. 
Ha:  Group  is  nonnormal. 

TEST  STATISTIC: 

TS:  W  =  0.953292 
alpha  =  .05 

REJECTION  REGION: 

W  <  0.850 

CONCLUSION: 

There  is  insufficient  evidence  to  reject  the  null 
hypothesis  for  the  SEV2  data  group.  Therefore  the  sample 
population  is  normally  distributed  for  the  purposes  of  this 
experiment . 
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Variable=SEVl 
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Figure  13.  SAS  Univariate  Procedure  for  SEVl 

Test  for  Normality  on  SEVl 

The  data  consist  of  a  random  sample  XI ,  X2  . . .  ,  Xn 

of  size  n  =  11  for  the  SEVl  group. 

ASSUMPTIONS:  The  sample  is  a  random  sample. 

HYPOTHESIS: 

Ho:  Group  samples  are  from  a  normal  population. 
Ha:  Group  is  nonnormal. 

TEST  STATISTIC: 


TS:  W  =  0.929086 
alpha  =  .05 
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REJECTION  REGION: 

'V  <  0.850 

CONCLUSION: 

There  is  insufficient  evidence  to  reject  the  null 
hypothesis  for  the  SEVl  data  group.  Therefore  the  sample 
population  is  normally  distributed  for  the  purposes  of  this 
experiment . 
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Figure  14.  SAS  Univariate  Procedure  for  LOC3 


Test  for  Normality  on  L0C3 

The  data  consist  of  a  random  sample  XI ,  X2  . . .  Xn 
of  size  n  =  14  for  the  L0C3  group. 
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ASSUMPTIONS:  The  sample  is  a  random  sample. 

HYPOTHESIS: 

Ho:  Group  samples  are  from  ’mal  population. 

Ha:  Group  is  nonnormal. 

TEST  STATISTIC: 

TS:  W  =  0.936639 
alpha  =  .05 

REJECTION  REGION: 

W  <  0.874 

CONCLUSION: 

There  is  insufficient  evidence  to  reject  the  null 
hypothesis  for  the  L0C3  data  group.  Therefore  the  sample 
population  is  normally  distributed  for  the  purposes  of  this 
experiment . 
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Figure  15.  SAS  Univariate  Procedure  for  L0C2 


Test  for  Normality  on  L0C2 

The  data  consist  of  a  random  sample  XI ,  X2  . . .  ,  Xn 

of  size  n  =  11  for  the  L0C2  group. 

ASSUMPTIONS:  The  sample  is  a  random  sample. 

HYPOTHESIS: 


Ho:  Group  samples  are  from  a  normal  population 
Ha:  Group  is  nonnormal. 
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TEST  STATISTIC: 


TS:  W  =  0.934284 
alpha  =  .05 

BEJECTION  BEGION: 

W  <  0.850 

CONCLUSION: 

There  is  insufficient  evidence  to  reject  the  null 
hypothesis  for  the  L0C2  data  group.  Therefore  tne  sample 
population  is  normally  distributed  for  the  purposes  of  this 
experiment . 
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Variable=LOC 1 
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Figure  16.  SAS  Univariate  Procedure  for  LOCI 


Test  for  Normality  on  LOCI 

The  data  consist  of  a  random  sample  XI ,  X2  . . .  ,  Xn 

of  size  n  =  11  for  the  LOCI  group. 

ASSUMPTIONS:  The  sample  is  a  random  sample. 

HYPOTHESIS: 

Ho:  Group  samples  are  from  a  normal  population. 
Ha;  Group  is  nonnormal. 
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TEST  STATISTIC: 


TS:  W  =  0.957196 
alpha  =  .05 

BEJECTION  BE6I0H: 

W  <  0.850 

CONCLUSION: 

There  is  insufficient  evidence  to  reject  the  null 
hypothesis  for  the  LOCI  data  group.  Therefore  the  sample 
population  is  normally  distributed  for  the  purposes  of  this 
experiment . 

Performing  an  Analysis  of  Variance 

It  is  necessary  to  perform  an  analysis  of  variance  on 
the  independent  data  groups.  All  the  data  has  proved  to  be 
normally  distributed,  meaning  all  data  groups  failed  to 
reject  the  null  hypothesis.  It  must  now  be  determined  if 
there  is  a  significant  difference  between  the  groups. 

Tables  3-5  give  the  name  of  the  dependent  variable 
compared,  the  degrees  of  freedom,  the  p-value  for 
significance  and  other  statistical  data.  The  p-value  is 
extremely  important  and  will  be  compared  to  the  reference 
probability  value  of  0.05  significance.  If  the  p-value  is 
lower  than  the  reference  p-value,  then  it  can  be  concluded 
that  the  mean  is  significantly  different  among  the 
different  display  groups;  three  dimensional,  two 
dimensional  and  the  standard  ABO  display.  Finally  by  using 
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the  Bonferroni  option  with  the  ANOVA  test,  the  data  groups 
will  be  compared  against  each  other.  The  output  will 
indicate  any  difference  between  each  pair  of  means.  It 
also  indicates  a  confidence  interval  for  the  difference, 
and  will  indicate  when  the  groups  are  different  by  three 
asterisks  {»»»).  Tables  3-5  show  the  SAS  ANOVA  output 
followed  by  a  hypothesis  test  to  either  accept  or  reject 
the  null  hypotheses  that  the  means  are  significantly 
di f  f  erent . 
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Table  3  -  ANOVA  Test  on  Correct  Dama>te  Groups 


General  Linear  Models  Procedure 
Dependent  Variable:  DAMAGE 


Source 

DF 

Sum  of  Squares 

F 

Value 

Pr  >  F 

Model 

2 

123.68759019 

4.11 

0.0254 

Error 

33 

496.20129870 

Corrected 

Total  35 

619.88888889 

R-Square 

C.  V. 

DAMAGE  Mean 

0. 199532 

27.80806 

13  , 

9444444 

Source 

DF 

Type  I  SS 

F 

Value 

Pr  >  F 

GROUP 

2 

123.68759019 

4.11 

0.0254 

Source 

DF 

Type  III  SS 

F 

Value 

Pr  >  F 

GROUP 

2 

123 . 68759019 

4.11 

0.0254 

Bonferroni  (Dunn)  T  tests  for  variable;  DAMAGE 

NOTE:  This  test  controls  the  type  I  experiment  wise  error 
rate  but  generally  has  a  higher  type  II  error  rate 
than  Tukey ’ s  for  all  pairwise  comparisons. 


Alpha=  0.05  Confidence=  0.95  df=  33  MSE=  15.0364 
Critical  Value  of  T=  2.52221 


Comparisons  significant  at  the  0.05  level 
are  indicated  by  ’»»«’. 


GROUP 

Comparison 

Simul taneous 
Lower 

Conf idence 
Limit 

Di f  f  erence 
Between 
Means 

Simultaneous 

Upper 

Conf idence 
Limit 

3  -  2 

3  -  1 

-1.415 

0.494 

2.526 

4 . 435 

6.467 

8.376 

2  -  3 

2  -  1 

-6 . 467 
-2.261 

-2.526 

1.909 

1 .415 
6.079 

1  -  3 

1  -  2 

-8.376 

-6.079 

-4.435 

-1.909 

-0.494 

2.261 
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Conducting  ANOVA  F  test  on  DAMAGE  Variable  group 
HYPOTHESIS: 

Ho:  The  means  of  each  group  are  equal. 

Ha:  at  least  two  treatment  means  differ. 

TEST  STATISTIC: 

TS:  F  =  4.11 
p-value  =  0.0254 

CONCLUSION: 

Reject  the  null  hypothesis  that  the  means  of  the 
sample  groups  are  equal.  Therefore  at  least  two  treatment 
means  differ.  By  looking  at  the  bonferonni  option,  the  two 
groups  that  differ  are  the  three  dimensional  group  and  the 
standard  ABO  group. 
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Table  4  -  ANOVA  Teat  on  Correct  Severity  Groups 
General  Linear  Models  Procedure 
Dependent  Variable:  SEVERITY 


Source 

DF 

Sum  of  Squares 

F 

Value 

Pr  >  F 

Model 

2 

10.38961039 

0.32 

0.7292 

Error 

33 

537.61038961 

Correct  Total 

35 

548.00000000 

R-Square 

C.  V. 

SEVERITY  Mean 

0 .018959 

40.36239 

10, 

,0000000 

Source 

DF 

Type  I  SS 

F 

Value 

Pr  >  F 

GROUP 

2 

10.38961039 

0.32 

0.7292 

Source 

DF 

Type  III  SS 

F 

Value 

Pr  >  F 

GROUP 

2 

10.38961039 

0.32 

0.7292 

Bonferroni  (Dunn)  T  tests  for  variable:  SEVERITY 


NOTE:  This  test  controls  the  type  I  experiment  wise  error 
rate  but  generally  has  a  higher  type  II  error  rate  than 
Tukey’s  for  all  pairwise  comparisons. 

Alpha=  0.05  Confidence^  0.95  df=  33  MSE=  16.29122 
Critical  Value  of  T=  2.52221 

Comparisons  significant  at  the  0.05  level 
are  indicated  by  '»»»’. 


Simul taneous 

Simul taneous 

Lower 

Di f  f  erence 

Upper 

GROUP 

Conf idence 

Between 

Conf idence 

Compar i son 

Limit 

Means 

Limit 

3  -  2 

-3 . 530 

0.571 

4.673 

3  -  1 

-2 . 803 

1 . 299 

5 . 400 

2  -  3 

-4 . 673 

-0.571 

3.530 

2  -  1 

-3.614 

0 . 727 

5 . 068 

1  -  3 

-5 . 400 

-1 . 299 

2 . 803 

1  -  2 

-5 . 068 

-0 . 727 

3.614 
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Conductinji;  ANOVA  F  test  on  SEVERITY  Variable  >troup 


HYPOTHESIS: 

Ho:  The  means  of  each  group  are  equal. 

Ha:  at  least  two  treatment  means  differ. 

TEST  STATISTIC: 

TS:  F  =  0.32 
p-value  =  0.7292 

CONCLUSION: 

Fail  to  reject  the  null  hypothesis  that  the  means  of 
the  sample  groups  are  equal.  Therefore,  none  of  the 
treatment  means  differ.  By  looking  at  the  bonferonni 
option,  there  are  no  groups  that  differ.  There  is  no 
significant  difference  in  the  sample  means  for  the  correct 
sever! ty . 
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Table  5 


ANOVA  Test  on  Correct  Location  Groups 


General  Linear  Models  Procedure 
Dependent  Variable:  LOCATION 


Source 

DF 

Sum  of  Squares 

F 

Value 

Pr  >  F 

Mode  1 

2 

57.33585859 

1 . 29 

0.2889 

Error 

33 

733.63636364 

Corrected 

Total  35 

790.97222222 

R-Square 

C.  V. 

LOCATION  Mean 

0 . 072488 

36.34704 

12 

.9722222 

Source 

DF 

Type  I  SS 

F 

Value 

Pr  >  F 

GROUP 

2 

57.33585859 

1 . 29 

0 . 2889 

Source 

DF 

Type  III  SS 

F 

Value 

Pr  >  F 

GROUP 

2 

57.33585859 

1 . 29 

0 . 2889 

Bonferroni  (Dunn)  T  tests  for  variable;  LOCATION 

NOTE;  This  test  controls  the  type  I  experiment  wise  error 
rate  but  generally  has  a  higher  type  II  error  rate  than 
Tukey’s  for  all  pairwise  comparisons. 

Alpha=  0.05  Confidence*  0.95  df=  33  MSE=  22.2314 
Critical  Value  of  T=  2.52221 

Comparisons  significant  at  the  0.05  level 
are  indicated  by 


Simul taneous 

Simultaneous 

Lower 

Di f  f erence 

Upper 

GROUP 

Conf idence 

Between 

Conf idence 

Comparison 

Limit 

Means 

Limit 

3  -  2 

-4.337 

0.455 

5 . 246 

3  -  1 

-1 .882 

2.909 

7.701 

2  -  3 

-5.246 

-0.455 

4 . 337 

2  -  1 

-2.616 

2.455 

7.525 

1  -  3 

-7.701 

-2.909 

1 .882 

1  -  2 

-7.525 

-2.455 

2.616 
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Conducting  ANOVA  F  test  on  LOCATION  Variable  ji?roup 


HYPOTHESIS: 

Ho:  The  means  of  each  group  are  equal. 

Ha:  at  least  two  treatment  means  differ. 

TEST  STATISTIC: 

TS :  F  =  1.29 
p-value  =  0.2869 

CONCLUSIOM: 

Fail  to  reject  the  null  hypothesis  that  the  means  of 
the  sample  groups  are  equal.  Therefore,  none  of  the 
treatment  means  differ.  By  looking  at  the  bonferonni 
option,  there  are  no  groups  that  differ.  There  is  no 
significant  difference  in  the  sample  means  for  the  correct 
location . 

Summary 

This  chapter  presented  a  great  deal  of  data  collected 
and  analyzed  for  determining  the  preference  of  the 
displays.  The  data  was  analyzed  for  normality  and  then 
compared  against  each  other  through  the  ANOVA  procedure  to 
see  if  the  sample  group  means  were  significantly  different. 
The  following  chapter  will  give  conclusions  and 
recommendations  from  this  analysis.  Chapter  V  will  also 
answer  all  the  investigative  questions  posed  in  Chapter  I. 
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V. 


Conclusions  and  Recommendations 


Overview 

The  previous  chapter  analyzed  the  data  compiled  from 
the  true  experiment  to  determine  if  there  was  a  significant 
difference  between  the  three  displays  --  three  dimensional, 
two  dimensional,  and  the  standard  ABO  Display.  This 
chapter  discusses  the  results  of  that  analysis  and  the 
possible  conclusions  and  recommendations  for  further 
research . 

Conclusions  (Investigative  Question  »  1) 

Investigative  question  •  1  asks:  Is  there  a 
significant  difference  with  using  two  dimensional  or  three 
dimensional  computer  animation  graphics  in  the  ABO 
decision-making  process? 

The  data  analyzed  in  the  methodology  chapter 
indicated  no  s i gn i f i cant  difference  between  using  two 
dimensional  or  three  dimensional  computer  animation 
graphics  in  the  ABO  decision-making  process.  However, 
there  was  a  significant  difference  between  using  the  three 
dimensional  graphics  display  as  opposed  to  using  the 
standard  ABO  map  technique.  The  significant  difference  was 
in  determining  the  correct  type  of  damage  indicated  on  the 
simulated  air  base.  The  significance  value  of  this 
difference  is  .025.  This  value  is  below  the  reference 
value  of  .05  for  a  notable  difference  therefore,  it  is 
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justified  to  report  this  difference.  It  is  possible  that 
there  might  have  been  a  notable  difference  between  the 
other  means  of  the  different  display  groups  if  more  data 
points  were  available  for  the  sample.  It  is  believed  by 
this  researcher  that  there  were  not  enough  data  points 
collected  in  the  experiment  for  a  significance  to  appear. 

Conclusions  (Investigative  Question  *  2) 

Investigative  question  *  2  asks:  What  ABO 
information  can  or  cannot  be  presented  with  each  type  of 
animation  graphic  display? 

After  inspecting  the  answers  obtained  from  the 
questionnaires  in  the  experiment,  it  was  noted  by  some 
subjects  that  the  two  dimensional  graphics  display  juld 
not  accurately  present  damage  which  occurred  to  a  facility 
on  its  side.  The  two  dimensional  graphics  display  could 
only  present  damaged  areas  which  appeared  on  the  tops  of 
facilities.  Subjects  who  used  the  two  dimensional  display 
had  difficulty  identifying  the  damaged  areas  located  on  the 
side  of  a  facility  and  thus  scored  poorly  because  of  it. 
They  did  however,  according  to  their  scores,  correctly 
identify  damaged  areas  which  were  on  the  side  of  a 
facility.  This  suggests  that  possibly  not  enough  side 
damaged  areas  were  entered  into  the  simulated  air  base  for 
a  significant  conclusion.  Because  the  means  did  not  differ 
significantly,  it  is  difficult  to  prove  otherwise. 
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Also  a  very  important  observation  to  note  was  the  use 
of  colors  for  both  displays.  The  use  of  bright  red  and  dark 
red  in  the  two  graphic  displays  was  a  mistake.  Subjects 
using  both  displays  repeatedly  remarked  about  having 
difficulty  in  determining  the  severity  of  a  damaged  area. 
This  is  justified  by  observing  the  mean  data  for  each 
group  on  correct  severity.  A1 .  the  means  appear  below  the 
other  areas  tested.  A  suggestion  was  made  to  change  dark 
red  to  yellow  to  represent  medium  severity  for  better 
identification. 

The  three  dimensional  graphic  display  was  able  to 
present  all  the  information  needed  for  the  mission  of  ABO. 
There  were  some  difficulties  as  to  location  of  a  damaged 
area.  For  example,  although  subjects  located  a  crater  on 
the  runway,  they  had  difficulty  deciding  if  it  was  in  the 
center  of  the  runway,  south  east,  or  south  west.  To  solve 
this  possible  problem,  a  grid  system  could  be  incorporated 
in  the  graphic  system  to  locate  damaged  areas  by  grid 
coordinates  instead  of  cardinal  references. 

Conclusions  (Investigative  Question  »  3) 

Investigative  question  *  3  asks;  What  time 
constraints  are  applicable  with  the  production  two 
dimensional  or  three  dimensional  animation  graphics  for  the 
decision  maker? 
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As  mentioned  earlier  in  the  literature  review, 


Chapter  II,  the  production  of  a  three  dimensional  computer 
graphic  animation  takes  some  time.  Each  damaged  facility 
animation  on  the  three  dimensional  display  took  an  average 
time  of  five  hours  to  generate  on  the  Amiga  2000  computer. 
The  Amiga  computer  processor  clock  speed  was  7.4  Mhz.  This 
is  comparatively  slow  compared  to  newer  Amiga  machines 
presently  running  at  speeds  up  to  50  Mhz.  With  this  new 
technology,  the  creation  of  a  graphic  animation  would  be 
reduced  ten  fold.  What  would  take  an  hour  to  produce  with 
the  old  model  machine  would  take  less  than  five  minutes 
with  the  newer  model.  Thus,  one  damaged  facility  animation 
would  take  approximately  twenty  five  minutes  or  less  to 
create.  Even  more  sophisticated  and  powerful  machines  in 
the  market  today  could  produce  an  animation  graphic  even 
quicker . 

A  two  dimensional  graphic  display  is  relatively  quick 
to  produce  since  only  one  picture  (the  top  down  shot)  is 
necessary.  The  time  it  takes  to  produce  a  damaged  facility 
picture  is  less  than  30  seconds.  This  time  constraint  is 
based  on  the  assumption  that  the  two  dimensional  graphic 
display  is  created  in  the  same  manner  in  which  it  was 
created  in  the  experiment.  Other  methods  of  two 
dimensional  picture  creation  could  be  used  as  described  in 
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Chapter  II  which  could  possibly  take  more  time.  The  time 
necessary  might  involve  human  (manual)  intervention  to 
touch  up  the  graphic  picture. 


Conclusions  (Investigative  Question  »  4) 

Investigative  question  •  4  asks:  What  finances  are 
associated  with  the  acquisition  of  a  computer  graphics 
display  system  and  will  it  be  cost  effective? 

The  finances  involved  in  acquiring  a  computer 
graphics  display  system  could  be  costly.  A  computer 
graphics  display  system  consists  of  what  was  mentioned  in 
Chapter  II:  A  powerful  computer  processor,  a  high 
resolution  graphic  display  monitor,  a  moderate  amount  of 
RAM  to  create  the  images,  and  any  graphics  input  and  output 
devices  needed.  Whether  those  costs  are  Justified  is  a 
difficult  question  to  answer.  Based  on  the  analysis  from 
Chapter  IV,  the  three  dimensional  display  did  prove  to  be 
significantly  more  effective  in  determining  the  type  of 
damage  a  facility  incurred  after  a  post-attack  from  enemy 
forces.  If  the  ABO  mission  or  any  other  organization’s 
mission  demands  that  this  be  critical,  then  the  costs 
associated  to  purchasing  a  computer  graphic  system  are 
justified.  It  is  also  extremely  important  to  note  that 
this  system  will  not  Just  be  used  for  one  application. 
Multiple  applications  are  apparent  for  this  type  of  system. 
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The  cost  effectiveness  of  a  three  dimensional  computer 
graphic  system  will  go  up  as  more  and  more  applications  are 
brought  to  its  use. 

Overall  Conclusions 

As  mentioned  before,  there  was  a  significant 
difference  noted  in  determining  the  type  of  damage  a 
facility  incurred  with  the  three  dimensional  graphic 
display  and  the  standard  ABO  map  display.  Although  there 
was  not  a  significant  difference  between  two  and  three 
dimensional  computer  animation  graphics,  the  data  analyzed 
indicates  a  preference  for  three  dimensional  computer 
animation  graphics  over  the  standard  ABO  display.  While 
observing  some  of  the  subjects  using  two  or  three 
dimensional  graphics,  the  researcher  noted  some  frustration 
with  the  subjects  using  the  two  dimensional  display.  This 
seems  to  be  due  to  the  lack  of  information  presented  with 
the  two  dimensional  picture.  If  we  assume  the  adage  that 
"More  is  better'  one  could  be  lead  to  believe  that  the 
three  dimensional  graphic  animation  is  better.  But  that 
fact  cannot  be  proved  conclusively  with  this  present  data. 

Recommendations  for  Future  Research 

Future  research  in  the  area  of  two  and  three  » 

dimensional  computer  animation  graphics  is  strongly 
advised.  Future  research  involving  the  preference  between 
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two  and  three  dimensional  graphics  in  other  applications  is 
recommended.  However,  as  mentioned  before  the  samples 
sizes  of  each  display  group  should  have  been  larger.  It  is 
recommended  that  future  research  be  conducted  with  a  larger 
sample  size  from  each  display  group.  A  sample  size  for 
each  group  should  be  about  30  or  more.  The  new  sample  size 
groups  might  show  a  more  defined  difference  between  the 
displays . 

If  similar  research  is  to  be  conducted  using  the  ABO 
decision-making  process  application,  then  correction  of  the 
problems  encountered  with  this  experiment  should  be 
accompl ished . 

Summary 

This  thesis  attempted  to  determine  the  preference 
between  two  computer  animation  displays  --  two  and  three 
dimensional.  By  using  the  ABO  decision-making  process  as 
an  environment,  three  displays  were  created.  Subjects  were 
then  randomly  selected  to  use  one  of  the  three  displays  and 
their  data  was  compared  and  analyzed.  The  results 
suggested  a  significant  difference  between  the  three 
dimensional  display  and  the  standard  ABO  display  currently 
being  used  for  determining  the  preference  in  a  particular 
display.  The  results,  however,  did  not  show  a  significant 
difference  between  two  and  three  dimensional  displays. 
Therefore  there  was  no  significant  preference  between  the 
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use  of 
on  the 


two  or  three  dimensional  computer  animation  displays 
air  base  operability  decision  making  process. 
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APPENDIX  A:  ABO  Damage  Listing  Report 


Damage  Type 

Severity 

Buildinf(  • 

Location 

UXO 

MEDIUM 

100 

North-West 

(Ground) 

CRATER 

MINIMAL 

103 

East  (Ground) 

UXO 

MEDIUM 

Runway  27 

East  End 

UXO 

MEDIUM 

106 

Center  (Ground) 

CRACK 

MEDIUM 

105 

Top  (Roof) 

UXO 

MEDIUM 

Hangers 

Between  1  &  2 

CRATER 

MINIMAL 

POL  Dome 

South  (Pad) 

UXO 

MEDIUM 

104 

East  (Ground) 

CRACK 

MEDIUM 

108 

North  (Wall) 

UXO 

MEDIUM 

no 

East  (Ground) 

CRATER 

MINIMAL 

no 

North  (Ground) 

CRATER 

CRITICAL 

Hangers 

•  3  South 

CRACK 

MINIMAL 

101 

West  (Wall) 

CRACK 

MEDIUiyI 

105 

North  (Wall) 

CRATER 

MINIMAL 

107 

West  (Ground) 

CRACK 

MEDIUM 

108 

South  (Wall) 

CRATER 

MINIMAL 

Runway  18 

North-West 

CRATER 

MEDIUM 

100 

South  (Wall) 

CRACK 

CRITICAL 

102 

East  (Wall) 

UXO 

MEDIUM 

106 

North-West 

(Ground) 

CRATER 

MINIMAL 

Hangers 

*1  East 

CRATER 

MINIMAL 

Runway  1 8 

South-East 
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Damage  Type 

Sever i tv 

Building  * 

Location 

CRACK 

CRITICAL 

104 

Inside  (Wall) 

CRATER 

MINIMAL 

107 

Top  (Building) 

CRACK 

MINIMAL 

109 

Top  (Building) 

CRATER 

MINIMAL 

Runway  27 

North-Center 

CRACK 

CRITICAL 

1 1 1 

North  (Wall) 

CRATER 

MINIMAL 

Runway  1 8 

West-Central 
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APPENDIX  B:  Air  Base  Operability  Questionnaire 


For  each  building,  identify  the  total  number  of 
damaged  areas,  the  type,  location  and  severity  of  each 
damaged  areas.  Some  buildings  may  have  damage,  some  may 
not,  others  may  have  more  than  one  damaged  area.  None  of 
the  facilities  will  have  more  than  three  damaged  areas. 

EXAMPLE : 

You  find  building  105  has  a  black  crater  on  its  south  wall. 
Look  for  the  question  number  corresponding  to  building  105 
and  fill  in  the  necessary  spaces  on  the  answer  sheet. 


10.  Building  105 

type  severity  LOCATION  AREA 


1 

=  CRATER 

1 

=  MINIMAL 

(BLACK) 

1 

= 

NORTH 

1 

=  WALL 

2 

=  CRACK 

2 

=  MEDIUM 

(DARK  RED) 

2 

= 

SOUTH 

2 

=  GROUND 

3 

=  UXO 

3 

=  CRITICAL  (BRIGHT  RED) 

3 

EAST 

3 

=  ROOF 

4 

WEST 

5 

TOP 

Starting  then  with  row  number  10,  the  following  is  filled. 


10  . 

(1) 

<  4^  J 

(3) 

(4) 

(3) 

For 

Type 

20  . 

(1) 

(2) 

(3) 

(4) 

(5) 

For 

Severity 

30. 

(1) 

(2) 

(3) 

(4) 

(5) 

For 

Location 

40  . 

(1) 

(2) 

(3) 

(4) 

(5) 

For 

Area 

For  location  you  may  answer  more  than  one  space,  for 
instance  a  crater  is  located  North-West  (NW)  therefore  you 
fill  in  North  and  West  together,  numbers  1  &  4 . 
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COLUMN  1 

COLUMN  2 

COLUMN  3 

COLUMN  4 

TYPE 

SEVERITY 

LOCATION 

AREA 

1  =  CRATER 

2  =  CRACK 

3  =  UXO 

1  =  MINIMAL  (BLACK) 

2  =  MEDIUM  (DARK  RED) 

3  =  CRITICAL  (BRIGHT 

)  =  NORTH 

2  =  SOUTH 
RED)  3  =  EAST 

4  =  WEST 

5  =  TOP 

1  =  WALL 

2  =  GROUND 

3  =  ROOF 

FOR 

ANSWER  SHEET 

1  . 

Building  100 

4  . 

Bui  1 d 1 ng  101 

7  . 

Bui Iding  102 

10  . 

Bui Iding 

103 

43  . 

Building 

104 

46  . 

Bui Iding 

105 

49  . 

Bui Iding 

106 

82  . 

Bui Iding 

107 

85  . 
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TYPE 


SEVERITY 


LOCATION 


AREA 


CRATER 

MINIMAL  (BLACK) 

NORTH 

WALL 

CRACK 

MEDIUM  (BROWN) 

SOUTH 

GROUND 

UXO 

CRITICAL  (BLACK) 

EAST 

ROOF 

WEST 

TOP 

For  Runway  27 

TYPE  SEVERITY  LOCATION  AREA 


For  Runway  18 

TYPE  SEVERITY  LOCATION  AREA 


For  Hangers  1-North,  2-Middle,  3-South 

TYPE  SEVERITY  LOCATION  AREA 


APPENDIX  C:  Frequency  Tables 


Table  6  -  Frequency  of  3D  Total  Responses 


T0T3 

Frequency 

Percent 

Cumulative 

Frequency 

Cumulative 

Percent 

8 

1 

7. 1 

1 

7 . 1 

13 

2 

14.3 

3 

21.4 

14 

1 

7. 1 

4 

28.6 

15 

1 

7 . 1 

5 

35.7 

16 

2 

14.3 

7 

50.0 

18 

3 

21.4 

10 

71.4 

19 

2 

14.3 

12 

85 . 7 

20 

1 

7 . 1 

13 

92.9 

23 

1 

7. 1 

14 

100 . 0 

Table  7  -  Frequency  of 

3D  Correct 

Damai^e 

Cumulative 

Cumulative 

DAM3 

Frequency 

Percent 

Frequency 

Percent 

8 

1 

7 . 1 

1 

7. 1 

13 

2 

14.3 

3 

21.4 

14 

1 

7. 1 

4 

28.6 

15 

2 

14.3 

6 

42.9 

16 

1 

7 . 1 

7 

50.0 

18 

4 

28 . 6 

1 1 

78 . 6 

19 

1 

7. 1 

12 

85.7 

20 

2 

14 . 3 

14 

100.0 

Table  8  -  Frequency  of  3D  Correct  Severity 


SEV3  Frequency 

Cumulative 
Percent  Frequency 

Cumulative 

Percent 

2 

1 

7. 1 

1 

7  , 

,  1 

6 

1 

7. 1 

2 

14 

.3 

8 

2 

14.3 

4 

28. 

,  6 

9 

1 

7. 1 

5 

35, 

.  7 

10 

2 

14.3 

7 

50, 

.0 

1  1 

1 

7. 1 

8 

57, 

.  1 

12 

1 

7. 1 

9 

64  . 

,3 

13 

1 

7. 1 

10 

71  , 

.  4 

14 

3 

21.4 

13 

92. 

.  9 

17 

1 

7. 1 

14 

100  , 

,  0 

Table  9  -  Frequency  of  3D  Correct  Locations 


L0C3 

Frequency 

Percent 

Cumulative 

Frequency 

Cumulative 

Percent 

e 

3 

21.4 

3 

21.4 

10 

1 

7.1 

4 

28.6 

13 

1 

7. 1 

5 

35.7 

14 

3 

21.4 

8 

57. 1 

15 

1 

7. 1 

9 

64.3 

16 

1 

7.1 

10 

71.4 

17 

1 

7. 1 

1 1 

78.6 

i 

1 

7. 1 

12 

85.7 

20 

1 

7. 1 

13 

92.9 

21 

1 

7. 1 

14 

100.0 

Table  10  -  Frequency  of 

2D  Total  Res 

ponses 

Cumulative 

Cumulative 

T0T2 

Frequency 

Percent 

Frequency 

Percent 

1 1 

1 

9. 1 

1 

9. 1 

13 

1 

9. 1 

2 

18.2 

14 

2 

18.2 

4 

36.4 

17 

2 

18.2 

6 

54.5 

18 

1 

9. 1 

7 

63.6 

19 

3 

27.3 

10 

90.9 

21 

1 

9. 1 

1  1 

100.0 

Table  11  - 

Frequency  of 

2D  Correct 

Damaf^e 

Cumulative 

Cumulati  V' 

DAM2 

Frequency 

Percent 

Frequency 

Percent 

10 

1 

9.1 

1 

9. 1 

1  1 

3 

27.3 

4 

36.4 

12 

2 

18.2 

6 

54.5 

14 

1 

9.1 

7 

63.6 

16 

3 

27.3 

10 

90.9 

20 

1 

9.1 

11 

100.0 
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Table  12  -  Frequency  of  2D  Correct  Severity 


SEV2 

Frequency 

Percent 

Cumulative 

Frequency 

Cumulative 

Percent 

5 

1 

9.1 

1 

9. 1 

6 

1 

9. 1 

2 

18.2 

B 

2 

18.2 

4 

36.4 

9 

2 

18.2 

6 

54.5 

10 

1 

9.1 

7 

63.6 

1  1 

1 

9. 1 

8 

72 . 7 

13 

1 

9. 1 

9 

61.8 

15 

1 

9. 1 

10 

90.9 

16 

1 

9. 1 

1 1 

100.0 

Table  13  -  Freauency  of 

2D  Correct  Locations 

Cumulative 

Cumulative 

L0C2 

Frequency 

Percent 

Frequency 

Percent 

3 

1 

9. 1 

1 

9. 1 

10 

1 

9. 1 

2 

18 . 2 

1  1 

2 

18.2 

4 

36 . 4 

12 

1 

9. 1 

5 

45.5 

13 

1 

9. 1 

6 

54 . 5 

16 

1 

9. 1 

7 

63.6 

17 

2 

18 . 2 

9 

81.6 

19 

1 

9. 1 

10 

90.9 

20 

1 

9. 1 

1 1 

100.0 

Table  14  -  Frequency  of  Standard  Total  Responses 


TOTl 

Frequency 

Percent 

Cumulative 

Frequency 

Cumulative 

Percent 

3 

1 

9. 1 

1 

9. 1 

7 

1 

9. 1 

2 

18.2 

9 

1 

9.  1 

3 

27.3 

1 1 

4 

36,4 

7 

63.6 

13 

1 

9.1 

8 

72.7 

17 

1 

9. 1 

9 

61.6 

19 

1 

9. 1 

10 

90.9 

21 

1 

9.1 

1 1 

100.0 
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Table 

15  -  Freauency  of  Standard  Correct 

Damajte 

Cumulative 

Cumulative 

DAMl 

Frequency 

Percent 

Frequency 

Percent 

3 

1 

9. 1 

1 

9. 1 

7 

1 

9. 1 

2 

18.2 

9 

2 

18.2 

4 

36.4 

10 

1 

9. 1 

5 

45 . 5 

1  1 

2 

18.2 

7 

63.6 

13 

1 

9. 1 

8 

72.7 

17 

1 

9. 1 

9 

81.8 

18 

1 

9. 1 

10 

90.9 

20 

1 

9. 1 

1 1 

100.0 

Table 

16  -  Freauency  of  Standard  Correct 

Severity 

Cumulative 

Cumulative 

SEVl 

Frequency 

Percent 

Frequency 

Percent 

2 

1 

9. 1 

1 

9. 1 

5 

3 

27.3 

4 

36.4 

7 

1 

9. 1 

5 

45.5 

1  1 

2 

18.2 

7 

63.6 

13 

3 

27.3 

10 

90.9 

17 

1 

9. 1 

1 1 

100.0 

Table 

17  -  Freauency  of  Standard  Correct 

Locations 

Cumulative 

Cumulative 

LOCI 

Frequency 

Percent 

Frequency 

Percent 

2 

1 

9. 1 

1 

9. 1 

7 

1 

9. 1 

2 

18.2 

9 

3 

27.3 

5 

45.5 

10 

1 

9. 1 

6 

54.5 

1  1 

1 

9. 1 

7 

63.6 

13 

1 

9.1 

8 

72.7 

14 

1 

9. 1 

9 

81.8 

18 

1 

9. 1 

10 

90.9 

20 

1 

9. 1 

1 1 

100.0 
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Figure  25.  Bar  Chart  of  ABO  Standard  Correct  Severity 
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Figure  26.  Bar  Chart  of  ABO  Standard  Correct  Locations 
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